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Abstract 

For cofinite Kleinian groups, with finite-dimensional unitary representations, we derive the Selberg trace 
formula. As an application we define the corresponding Selberg zeta-function and compute its divisor, thus 
generalizing results of Elstrodt, Grunewald and Mennicke to non-trivial unitary representations. We show 
that the presence of cuspidal elliptic elements sometimes adds ramification point to the zeta function. In fact, 
if O = ^[—^ + "^^^] is the ring of Eisenstein integers, then the Selberg zeta-function of PSL(2, O) contains 
ramification points. 
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Chapter 1 
Introduction 



The Selberg theory (Selberg trace formulas, Selberg zeta-hinctions, and related applications) has 
been well studied in both the two-dimensional scalar case ( IIIwa021 ) and the two-dimensional vec- 
tor case^ ( IIRoe66l , IVen82l , | Hej83| , IFis87l ). By "Two-dimensional vector case" we mean: cofinite 



Fuchsian groups with finite-dimensional unitary representations, and the "Scalar case" refers to 
the case with the trivial representation. Elstrodt, Grunewald and Mennicke extended the Sel- 
berg theory to the three-dimensional scalar case in IEGM98I . By the "Three-dimensional case" we 
mean: cofinite Kleinian groups. The main goal of this thesis is to extend the Selberg theory to the 
three-dimensional vector case. 

In this thesis we derive the Selberg trace formula for cofinite Kleinian groups^, with finite- 
dimensional unitary representations. As an application we define the corresponding Selberg zeta- 
function and compute its divisor, thus generalizing results of Elstrodt, Grunewald and Mennicke 
IIEGM98I to non- trivial unitary representations. 

Much of the two-dimensional vector and three-dimensional scalar cases extends in a straight 
forward manner to the three-dimensional vector case. However, the extension of several parts of 
the Selberg theory are more subtle in the three-dimensional case, especially in the vector case. One 
reason for this is because the set of finite-dimensional unitary representations of a fixed cofinite 
Kleinian groups is not well understood. Another reason is related to the structure of the stabilizer 
subgroup of a cusp. In the two-dimensional case the stabilizer subgroup of a cusp is a purely 
parabolic group that is isomorphic to a rank-one lattice, while in three dimensions the stabilizer 
subgroup of a cusp is a non-abelian group that contains elliptic elements, with a finite-index purely 
parabolic subgroup that is isomorphic to a rank-two lattice. The presence of elliptic elements in the 
stabilizer subgroup introduces some subtleties to the three-dimensional vector case, particularly in 
the computation of the divisor of the Selberg zeta-function. In addition, the fact that the stabilizer 
subgroup (in the three-dimensional case) contains a rank-two parabolic subgroup forced us to 
prove some additional estimates involving two-dimensional lattice sums. 

A Kleinian group is a discrete subgroup of PSL(2, C) = SL(2, C) /{±/}. Each element of PSL(2, C) 



^The works |Roe66 |, | Hej83|, |Fis87|, contain not only the two-dimensional vector case, but also its generalization, the 
case of unitary multiplier systems of arbitrary real weight. 

Kleinian groups is referred to in some texts as a discrete group of isometries acting on hyperbolic three-space, or a 
discrete subgroup of PSL(2, C) . 
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CHAPTER 1. INTRODUCTION 



is identified with a Mobius transformation, and has a well-known action on hyperbolic three-space 
H"^ and on its boundary at infinity— the Riemann sphere (see [EGM98, Section 1.1]) . A Kleinian 
group is cofinite iff it has a fundamental domain C H'^ of finite hyperbolic volume. 
We use the following coordinate system for hyperbolic three-space, 

m^={{x,y,r) £R^\r> 0}={(z,r) |z G C, r > 0}={z + rj G | r > 0}, 

with the hyperbolic metric 

, n dx'^ + dv'^ + dr'^ 
ds^ = ^ , 



and volume form 



dx dy dz 
dv = . 



The Laplace-Beltrami operator is defined by 

A = -r^{— — —) r— 
dx"^ dy^ dr"^ dr ' 

and it acts on the space of smooth functions / : H-^ V", where is a finite-dimensional complex 
vector space with inner-product { , )v- 

Suppose that F is a cofinite Kleinian group and x G Rep(r, V) (Rep(r, V) is the space of finite- 
dimensional unitary representations of F in P^). Then the Hilbert space of x—ciutomorphic functions 
is defined by 

n{r, x) ={f:M^^v\ fi^p) = xh)f{P) V7 e r, 

P e , and (/, /) ^/^ {f{P)J{P))v dv{P) < oo}. 

Here .F is a fundamental domain for F in H-^, and ( , )y is the inner product on V. Finally, let 
A = A(r, x) be the corresponding positive self-adjoint Laplace-Beltrami operator on H{r, x) ■ 

Our first result is the spectral decomposition of A on 7i(I^, x) (see Theorem 13.8.111 . Except for 
one important point, the proof of the spectral decomposition theorem is analogous to the two- 
dimensional vector and three-dimensional scalar cases. The one important point being, singularity 
at a cusp. To the best of the author's knowledge, prior to this thesis, the notion of singularity at 
a cusp was only defined for cofinite Fuchsian groups |Sel56J IRoe66l fVen82ll |Hej83| . In ijS.ll we 
extend the notion of singularity to cofinite Kleinian groups. 

In i l4.1l we give an explicit form of the Selberg trace formula for cofinite Kleinian groups with 
finite-dimensional unitary representations (see Theorem 13. 7.1| |. The new feature in the trace for- 
mula is a term of the form, 

K dime V 



The above term comes about from regularity at a cusp, and its value is computed using Kronecker's 
second limit formula ( see i j4.2l l. 

As an application of the spectral decomposition theorem we derive an identity involving con- 
jugacy relations of cuspidal elliptic elements. This identity is used in the proof of the Selberg trace 
formula and to show that under certain conditions, the Selberg zeta function admits a meromor- 
phic continuation (see Lemma [4.4.4l for the identity). 
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For Re(s) > 1 the Selberg zeta-function Z{s, T, x) is defined by the following product 

dime V 

Z{s,r,x)^ n n n {l-ijaiToy'^oF^NiTor^-^). 
{Tojen j=i i,k>o 

c{T,j,l,k) = l 

In !15.1l we introduce the various definitions and notations that are needed in order to define the 
zeta function, and meromorphically^ continue Z{s,T,x) to Re(s) < 1 while computing its divi- 
sor. The main new difficulty is handling the contribution of the cuspidal elliptic elements to the 
topological (or trivial) zeros and poles of Z(s, F, x). We show the following in i)5.3l 

Corollary. Let F — PSL(2, Z[— ^ + -^^-^]), and x = 1 (the trivial representation). Then Z{s, F, x) is not 
a meromorphic function'^ (it is the 6-th root of a meromorphic function). 

In addition, the methods of imply that the Selberg zeta-function of the Picard group is 
meromorphic: 

Corollary. Let F = PSL(2, Z[y/^]),and let x G Rep(r, V) . Then Z{s, F, x) is a meromorphic function. 



^We give the meromorphic continuation for certain cases, and for others show that the zeta function is a rational root 
of a meromorphic function. 

*This is the first example that the author is aware of where the Selberg zeta-function is not meromorphic. 



Chapter 2 

Prerequisite Material 



In order to make our presentation self-contained, we present some well-known results concerning 
hyperbolic three-space, the Laplace-Beltrami operator, and cofinite Kleinian groups. For more 
details see IEGM98I . 

2.1 Hyperbolic three-space 

Let denote the upper half space (of R'^) model of hyperbolic three-space. The space is 
parametrized by the following coordinates: 

lf={{x,y,r) eM.^ \ r> 0}={{z,r) | z G C,r > 0}. 

A point P e H'^ will be denoted by {x,y,r), {z,r), or z + rj. The standard hyperbolic metric^ 
hyperbolic metric and volume form are written respectively as 

2 dx^ + dy^ + dr^ dxdy dz 
ds = and dv = . 

Remark 2.1.1. An alternate model of hyperbolic three-space is the open three-ball, ={{x,y, z) G 
\ x^ + y"^ + z^ < 1}. When equipped with the metric 

^ dx^ + dy^ + dz^ 

(1 — — y2 _ 2;2)2 

is isometric to . 

The boundary (at infinity) of H'' can be realized (by Remark l2.1.1|l as the Riemann sphere P = 

CUoo. 

The Laplace-Beltrami A operator lies at the heart of this thesis. In our coordinates it can be 
written explicitly as the following differential operator^: 

A = —r( 1 \ ) + r — . 

dx'^ 9j/2 Qj.2 Qj. 

^The metric with -1 sectional curvature. 

^In our notation Z\ is a positive self-adjoint operator 
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CHAPTER 2. PREREQUISITE MATERIAL 



The orientation preserving isometry group of H'' can be identified with the group PSL(2, C) 
SL(2, C)/{±/}. Each element 

M= f " 5 ) ePSL(2,C) 



and t 



c d 

acts on M"^ as follows: AI{z + rj) = w + tj, where 

(az + b) (cz + d) + acr^ 

|cz + dp + |cpr2 |cz + dp + |cpr2 

The element A/ also acts on the boundary at infinity of H'^ via standard Mobius action on P, 

cC + d 

for C e F. 

2.2 Harmonic Analysis on 

For P = z + rj, P' = z' + r'j e H'^ denote by d{P, P') the (hyperbolic) distance in H'^ between P 
and P', and let (5(P, P') be defined by 

5{P,P')^- 



2rr' 

It follows that cosh((i(P, P')) = 6{P, P'), and that (5 is a point-pair invariant^. 

We can use the concept of a point-pair invariant to construct the resolvent kernel for A . For 

s eC,t > 1, set 

(^ + ^/^^)-^ 



Lemma 2.2.1. hEGM98\ Lemma 4.2.2] Let u e Cl{U^), s e C and \ = 1 - s'^. Then for all Q eU^ 



<Q) = ^ / ^s{5{P, Q))(A - \)u{P) dv{P). 



(2) The point-pair invariant (pso6 is the resolvent kernel for A on the Hilbert space of square integrable 
functions on 'B? . 

Let 5 = 5[l,oo) denote the Schwartz space of smooth functions 
k : [l,oo) C that satisfy Imix^ao x"'k^'^'> (x) = for all n,m € N>o- For each k g S, K=k o S 
is a point-pair invariant and the kernel of an operator K. : ^[^^^(IHI'^) i-^ Lf^{W^) defined by 



/C/(P)= / KiP,Q)fiQ)dv{Q). 



We have 



point-pair invariant is a map / : H'^ x H-^ ^ C defined almost everywhere satisfying f{MP, MQ) = f(P, Q) for 
all P, Q G H3, M G PSL(2, C). 



2.3. KLEINIAN GROUPS 
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Lemma 2.2.2. liEGM98\ Lemma 3.5.3] Let k e S, K{P, Q) = k{5{P, Q)) for P, Q e / : C 
a solution of Af = Xf, A = 1 — s^, and 

m .H,-,'>.lfk (i (< + i)) ,f - 1-) (< - 1) f . ,2.2.1, 

Then \2.2.2\ converges absolutely and 

[ K{P,Q)f{Q)dv{Q)=h{X)f{P). 

The lemma above says that if / is an eigenfunction* of A with eigenvalue A then / is also 
an eigenfunction of K, with eigenvalue h{X) depending only on the eigenvalue A and not on the 
particular eigenfunction. 

The map sending fc G 5 to holomorphic function h above is called the Selberg-Harish-Chandra 
transform of k. 

2.3 Kleinian Groups 

A subgroup r < PSL(2,C) is called a Kleinian group if for each _P e H'^ the orbit TP has no 
accumulation points in H^. An equivalent formulation is that F is a discrete subset of PSL(2, C) in 
the topology induced from C"' . 

A closed subset c H'^ is called a fimdamental domain of F if 

• meets each F— orbit at least once, 

• the interior JF° meets each F— orbit at most once, 

• the boimdary of J- has Lebesgue measure zero. 
For each Q e H'^ the set 

Vq (f) ={p e h3 I d(P, Q) < d{jP, Q) V 7 e r} 

is a fundamental domain for F that is centered at the point Q. 

We say that F is cocompact if it has a fundamental domain T that is compact, and cofinite^ if it 
has a fundamental domain T with 

vol(r) = j dv < oo 

(dv is the volume form of H'^). 



*The function / need not be in any Hilbert space. 
^Note tliat cocompact groups are also cofinite. 
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CHAPTER 2. PREREQUISITE MATERIAL 



7=1;!!; ) ePSL(2,c) 



2.4 Mobius Transformations 

Each element 

c d 

falls into exactly^ one of the following categories: 

• parabolic if | tr(7)| = 2 and tr 7 e R, 

• elliptic if < I tr(7)| < 2 and tr7 G M, 

• loxodromic if it is neither elliptic nor parabolic. 

There is a useful geometric characterization of the above notions. An element 7 e PSL(2, C) is 

• parabolic^ iff it has exactly one fixed point in P, 

• elliptic iff it has two fixed points in P and fixes the geodesic line in connecting the two 
points. 

• loxodromic iff it has two fixed points in P and has no fixed points in M'^ . 

2.5 Stabilizer Subgroups and Cusps 

Let r be a Kleinian group. For each Q e H"^ U P the stabilizer subgroup of Q is denoted by 

Tq ={ 7 e r I 7Q = Q } and for c e P, 

={7 e I 7 is parabolic or the identity }. 

Set 



S(C): 



a b 




0^aeC,beC} /{±I} < PSL(2, C), 



N{C) = 



1 b 
1 



beC W{±/} < PSL(2,C). 



The group B{C) is the stabilizer subgroup of PSL(2, C) that fixes 00 and N{C) is the maximal 
parabolic subgroup of B{C). It contains all parabolic elements fixing 00, and the identity element. 

A point C G P is called a cusp of P if P^- is a free abelian group of rank two. The set of cusps 
is denoted by Cr- Two cusps, a, and/3 of P called equivalent or P— equivalent if^ a £ P/3. The 
equivalence class of cusps is denoted by P \ Cr . 

The following is well known (see LEGM981 Chapter 2]). 

Lemma 2.5.1. Let T be a Kleinian group. 



^The identity element is an exception. Its trace is ±2 but it is not usually thought of as a parabolic element. 
^See the previous footnote. 

denotes the orbit of the point /3 G P. 



2.6. COFINITE KLEINIAN GROUPS 
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1. IfT contains a parabolic element then T is not cocompact. 

2. IfT is cofinite and is not cocompact then T contains a parabolic element. 

3. IfT is cofinite, ( £F, and contains a parabolic element then ( is a cusp ofT. 

4. IfT is cofinite then T has only finitely many T— equivalent classes of cusps. 

Next we study the structure' of the stabihzer subgroup of a cusp for cofinite Kleinian groups. 

Lemma 2.5.2. ^EGM98l Theorem 2.1.8 part (3)] Let T be cofinite with a cusp at oo. Then is a lattice 
in B{C) « C and one of the following three holds. 

2. Too is conjugate in B{C) to a group of the form 

) 0^6eA,6e{l,z}|/{±/} 

where A < C z's an arbitrary lattice. As an abstract group Too is isomorphic to I? x Z/2Z where the 
nontrivial element of'L/2'L acts by multiplication by —1. 

3. r'^ is conjugate in B{C) to a group of the form 

be On 



r(n,i) 



e eb 

e-i 



e expl^) forl<v<2n 



/{±I} 



where n = 4 or n = 6 and t\n and where On is the ring of integers in the quadratic number field 
Q (exp (^)) • Hence , as an abstract group Too is isomorphic to the group 1? xi TLlmLfor some 
m e {1, 2, 3, 4, 6}. An element 

e G Z/mZ ^ j exp ( — j |w e Z 

acts on I? ^ C^, by multiplication with t^, where m' = 4 in case 
me {1,2,4} and m' — 6 otherwise. 

2.6 Cofinite Kleinian Groups 

Let r be a cofinite Kleinian group. By Lemma 12.5.11 F has finitely many equivalence classes of 
cusps. 

Notation 2.6.1. Unless otherwise noted F is a cofinite Kleinian group with 

«;=|F\Cr|, 

and a maximal set {Cq}S=i of representatives for the equivalence classes of cusps. We set F^ = F^^ 
andF^ = F;.^. 



'The following structure theorem is extremely important, and without it we would be unable to extend the Selberg 
theory to cofinite Kleinian groups. 
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The following is elementary. 

Lemma 2.6.2. Let be a cusp ofT. Then there exists Bo, G PSL(2,C) and a lattice = Z 
Ztq,, Ini(TQ) > Q satisfying the following. 

2. BaTaB^^ acts discontinuously on P\{oo} = C. 
3. 



1 h 
1 



^ 5 e 



Notation 2.6.3. For each cusp class a = 1 . . . k we fix B^ and A^ from Lemma 12.6.21 and set 

= B^T^B-\ and T'^^= B^T'^B-\ 

While the action of T'^ on C is exactly the action of the (additive) lattice Aq, on^" C, the action 
of is a combination of the lattice action of A^ and possibly finite ordered euclidean rotations 

of C.°° 

The fundamental domain for a cofinite Kleinian group can be realized as the union of a compact 
(hyperbolic) polyhedron and k cusf sectors. Let Va C C be a fundamental domain^^ for the action 
of Fq^ on C, and let V'^ be the fundamental parallelogram with base point at the origin of the 
lattice A„. For r > set 

T^{Y)={z + rj\zeVo.,r>Y} 
and define the cusp sector, Ta{Y) = B^^Ta{Y). 

Lemma 2.6.4. KGM981 Prop. 2.3.9] Let F < PSL(2, C) be cofinite with k = |F \ Cr| . Then there exist 
Y > and a compact set Ty c H'^ such that 

T = TY\JT^{Y)\J---\JTrXY) 

is a fundamental domain for F. The compact set Ty can be chosen such that Ty n TaiYa) all are contained 
in the boundary of Ty and hence have Lebesgue measure 0. Also, J-a(i^a) H TfjiYp) = if a ^ (3. 



2.7 Cuspidal elliptic elements 

Throughout this section F is a cofinite Kleinian group. 

Lemma 2.7.1. Let F be cofinite. Then F has only finitely many elliptic conjugacy classes in F. 

Proof. Assume not. Then there is an infinite sequence of elliptic F— conjugacy classes {[e„]}. Next 
for each n choose representative e„ which fixes a point P„ on the boundary of the fundamental 
domain of F given in Lemma 12.6.41 Since F is discrete, the points must accumulate at least at one 
cusp. After conjugating F and passing to a subsequence of {Pn} we may assume that P„ — > oo. 

i^For 2 G C, we have ^ ^ z = z + b. 

^^The domain Va is a euclidean polygon. 



2.7. CUSPIDAL ELLIPTIC ELEMENTS 
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An application of IIEGM98[ Corollary 2.3.3] implies that e„ e Too - In short we have constructed in- 
finitely many elliptic F— conjugacy classes with a representative fixing the cusp oo. An elementary 
computation using Lemma 12.5.21 shows that there are only finitely many elliptic Too— conjugacy 

□ 



classes, a contradiction. 
Notation 2.7.2. For each a e {I 



i} set 



T = 



If 



then for some non-zero natural number iV, = 1 . We have the following application of Lemma l2.5.2l 

Lemma 2.7.3. The set is isomorphic to a finite subgroup of the unit circle S^, cyclic, and isomorphic 
to Fooa/r^ . Any element 7 e Fq, can be written uniquely in the form 7 = a[3, where a € Tq and 

Definition 2.7.4. An elliptic element 7 e F, is said to be cuspidal elliptic if at least one^^ of its fixed 
points in P is a cusp of F. Otherwise it is called a non-cuspidal elliptic element. The set of cuspidal 
elliptic elements is denoted by F^"^ . 

Notation 2.7.5. For a e { 1 . . . k} define CE^ to be the set of elements of F which are F-conjugate to 
an element of Fq, \ F^. We fix representatives of C£a :<?",..., ^^'^ define qf = BagfB^^ 



Since fixes 00, 

/ ef efw: 

where ef is a finite-ordered root of unity and wf e . 



(2.7.1) 



^^Actually by IEGM98I Cor 2.3.11] , if one fixed point is a cusp, then the other fixed point is also a cusp. 



Chapter 3 

The Spectral Decomposition Theorem 



In this chapter we prove the spectral decomposition theorem for cofinite Kleinian groups with 
finite-dimensional unitary representations. While much of our work is analogous to the two- 
dimensional vector and three-dimensional scalar cases^ there are some new complications that 
arise because of cuspidal elliptic elements^. 

3.1 Unitary Representations 

Throughout this section F is a cofinite Kleinian group with k — |r \ Cr| and cusp representatives 

{Ca}S=l- 

Let V be an n-dimensional complex vector space, with an inner product (•, •)y linear in the first 
argument and anti-linear in the second argument. For v £ V its norm is given by {v. v)y 
and the norm of a linear operator L -.V ^ V \s defined as 

in (\Lv\v\ 

L y = sup — — . 

vev V Mv J 

Definition 3.1.1. Define Rep(r, V) to be the set of pairs (x, V) where V a finite dimensional com- 
plex inner product space and x is a unitary representation of F in GL{V). 

We will abuse notation and identify x with (x, V^)- 

Definition 3.1.2. Let F be cofinite, k = |F \ Cr| and x e Rep(F, V) . For each a e {1, • • • , k} define 

Vo,={veV\x{l)v = v V7eFa}, 
V:^={v^V\x{l)v^v V7eF^}, 
ka = dim Va , 
la = dim , and 

k = k{r,x) = ^ka. 

a=l 

iWe encourage the reader to first read IEGM98I Sections 4.1,4.2,4.3,6.1,6.2,6.3,6.4] and IVen82| Chapter 1,2,3] 
^Fuchsian groups do not have cuspidal elliptic elements. 
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CHAPTER 3. THE SPECTRAL DECOMPOSITION THEOREM 



The subspace Va is called a singular subspace while is called an almost singular' subspace, 
ka is called the degree of singularity of the representation x at the cusp a, and k = fc(r, x) the degree 
of singularity of T relative to x- 

Definition 3.1.3. A representation x is singular at the cusp (a if ka > 0, regular at if fc^ = 0. It is 

singular if fc > 0, and regular if A; = 0. 

Unitary representions of the stabilizer subgroup 

Lemma 3.1.4. There exist Ea,Ra,Sa & with the following properties: 

(l)Ta = {E^Ri^Sl |0 < fc < rria, i,j € Z}. Here Ra, Sa are parabolic elements with BaRaB^^iP) = 

P +1 and BaSaB-^{P) = P + (here A^, = Z e Zrjfor all P e H^, and is elliptic of order ma- 
il) r'„ = {i?j,5^M,jeZ}. 

(3) The elements Ra and Sa commute but the group Ta is not abelian when > 1. 

(4) If in addition, rua > 1, then xi^a) maps onto itself. Furthermore, there exists a basis ofV^ so 
thatx{Ea)\v^ is diagonal. 

Proof. (1), (2), and (3) readily follow from IIEGM98I Theorem 2.1.8]. We prove (4): sei E = Ea,R = 
Ra,S ^ Sa- Since ERE^^ and R^^ are both parabolic and in r'„ it follows that the A = E-^R-^ER€ 
r'a is parabolic. Since A is parabolic, A — R^S^ for some i,j £ Z (applying (2)). By definition, the 
restriction x(^) — Iv^ - We have 

x{E)xiR)^x{R)x{E)x{A)- 
Next applying an arbitrary v E Va we obtain 

xiE)v = x{E)xiR)v = x{R)x{E)x{A)v = x{R)x{E)v (3.1.1) 

Thus x{R) fixes x{E)v and similarly x{S) fixes x{E)v, so by definition x{E)v G Va - Since x{E) is 
unitary, its restriction to Va is also unitary, hence Va has a diagonalizing basis. □ 

We remark that for a cusp a of F, the group is abelian. Thus, the unitary representations x 
restricted to F^ is diagonalizable and can be thought of as a direct sum of one-dimensional imitary 
representations of F^ . 

3.2 Automorphic Functions 

The set of F-automorphic functions ^(F, x) is the set of all Borel-measurable functions f -.V? ^ V 
that satisfy f{jP) — x{l)f{P) for all 7 G F. Such functions are uniquely determined by their 
values on a fundamental domain. With this in mind we fix a fundamental domain T C H'^ for F 
and define 

■H{r,x) = L2{T,V,dv,x)^{f ^ A(T,x) I j ^{f{P)J{P))v dv{P) < 

For /, g e x) define the inner product (/, g) = fjr{f{P),g{P))vdv{P)- With this inner product 
^■{r, x) is a Hilbert Space. For n G N let C"(H^, x) be space of F-automorphic functions which 
are n- times differentiable on H'^. 



■^The notion of almost singularity does not occur in ttie two dimensional case. 



3.3. FOURIER SERIES EXPANSION AT A CUSP 
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The Automorphic Laplacian Z\(r, x) 

Recall that the Laplace-Beltrami operator A is defined on twice continuously differentiable func- 
tions / ; H3 ^ C. Thus A can be defined on the set A{T, x) n C^iW.^, V, x), and for / e A{T, x) n 
C^{W^, V, x) it follows (since A commutes with isometries) that Afe A{T, x) n C^iM^, V, x)- 

Consider the dense subspace D = {f e n{r, x) nC^{m^, x) | Z\ / e H(r, x) }■ K follows that 
A : D ^ Ti.{r,x) is essentially self-adjoint and has a unique positive self-adjoint extension to a 
space D (see l,EGM98[ Section 4.1]). 

Definition 3.2.1. Let F be a cofinite Kleinian group and x G Rep(r, V) . We define the automorphic 
Laplacian 

AiT,x)-D^H{r,x) 
to be the self -adjoint extension of A : D ^ Ti.{r,x) ■ 

When there is little possibility for confusion, we will identify A with A{T,x)- The dense sub- 
space D can be realized as the set of functions in H{r, x) whose distributional Laplacian is in 

n{r,x)- 



3.3 Fourier series expansion at a cusp 
The scalar case 

Let / e A{T, 1) (that is / : ^ C, and for all P e 7 e T, /(7P) = /(P)), and let Ca be a cusp 
of r with B^^oo = (a- Then it is easily seen that / o e A{BaTB~^ , 1), in particular / o B~^ G 
AiBaT'^B-^,!). Since B^T'^B-^ is isomorphic to the lattice A„, / o B'^^P + w) = / o B-^{P) for 
all P G and w e A^. That is, / o is invariant imder that lattice Aq,. If we suppose further 
that / is smooth then we can expand / o B^^ as 

/(p-i(z + n))= .9pWe'"<^'^>- 

Here ( , ) is the (real) euclidean inner product in = C, A° is the lattice dual to A^. If we assume 
still further that A f ~ A/, and f{B^^{z + rj)) — 0{r^) for some e N, then a simple separation 
of variables argument (see IIEGM98i Section 3.3]) shows that for A = 1 — s^, s ^ 0, satisfies the 
Bessel equation 

- + A - 4^2|m|V2) g,ir) = 0, (3.3.1) 
whose general solution is ^^(r) = a^r/Cg (27r|^|r) + b^rls{27r\fi\r). Here 

^-^ TO!2'*+^™i s + m + 1 zsmsTT 

m— ^ ^ 

The fimction Ks{x) decreases exponentially, and Is{x) increases exponentially as a; — > 00. Apply- 
ing the growth bound of / we obtain 

f{B-\z + rj))^aor^+'+bor^-'+ ^ a^rK,{2TT\fi\r)e^^'<'^^'l (3.3.2) 
We will need the following fact later on: 
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Lemma 3.3.1. Let pbe a non-negative integer. Then 



dP 
drP 



^ a^rifs(27r|/i|r)e' 



for some constant \c\ > 0. 
Preliminary Lemmas 

Our goal in this section is to come as close as possible to diagonalizing the group of unitary trans- 
formations {xiTa)}- Recall that 



For each cusp (a of F let (/ — Pq) : V ^ denote the orthogonal projection onto V^. By 
definition of singularity, for each ^ v E there exists 7 G Fq satisfying xil)"^ 7^ 
Fix a cusp C,a of F. We can partition n — dim V as follows: 



We will next chose a particular basis of V that respects the partions above and is a close as possible 
to diagonalizing {x{Ta)}- 

Recall from Lemma l ISTl that r„ = {E'^W^S^ | < fc < m^, j G Z}. Here Ra,Sa are 
parabolic and Ea is elliptic. 

Let Bsa ={vai, ■ ■ ■ , Vaka, } t>e an orthonormal basis for Va- By lemma | |3.1.4t there exists a set 
Baa ={va{k^+i}, • ■ • j ^ai^} of orthouormal elements in V, pairwise orthogonal to the elements of 
Bsa so that the following conditions are satisfied: For each v G Baa,x{l)'" = for all 7 G F^, 
and x{Ea)v = Xv, where here Ea is the primitive elliptic element in F^ chosen in lemma l|3.1.4|l , 
and 1 7^ A G C with A™° — 1. Finally, there exists a set Bra ={va(ic,+i)j ■ • ■ i Van} of orthonormal 
elements of V, pairwise orthogonal to the elements of Bsa U Baa so that the following conditions 
are satisfied: For each v G Bra, x{l)v = A'u for each 7 G A'^ and some 1 7^ A' G C with | A'| — 1. We 
have proven the following. 

Lemma 3.3.2. For each cusp (a let F„ = { E^K^Si \ < k < nia, i,j E Z} (see lemma (|3T4l l). Then 
there exists an orthonormal basis 



of V with the following properties: 

(1) For each Vai G Bsa (1 < ' < ^q), xil)vai = Vai for all ^ G Ta- 
il) For each Val € Baa (fca + 1 < ^ < la), x{Ra)Val = X{Sa)Val = Vai, and x{Ea)Val = ^alVal- 

Here l^\aiE<C and A™;° = 1. 

(3) For each Vai G + 1 < ^ < n), x{Ra)vai = XRa,iVai, and x{Sa)vai = ^s^i'^ai- Here 

Xr^i,Xs^i G C, with |Afl^;| = |As„z| ~ 1, and Xb,^i,Xs^i are not both equal to one. 



Va={^eV\x{l)v^v V7GF„}, 



={« G 1/ I x(7)« = « V7GF:,}, 
ka = dim Va , and 
la = dim . 




{Val, 



Vak^,Va{k^ + l), ■ ■ ■ , ^al a, Va(U+l), ■ ■ ■ ,Van\ = Bsa U Baa U -B, 
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For each cusp Ca and + 1 < ^ < ?i fix O^^i and 9s^i so that 

^2^^en^, ^ ^^^^ ^2.^es^^ ^ (3 3 3) 

We remark that Bsa and Bj-a are analogous to the two dimensional singular and regular cases 
respectively (see I Ven82ll ). We call Baa the almost singular case. 

Let Pal be the orthogonal projection onto the subspace generated by Vai ■ 

Lemma 3.3.3. Let f e A{T, x)- 

(1) Ifl <l<la, then 

Paif{B-\Q + uj)) = Paif[B-^Q) 

for all Q e and uj e A^. 

(2) If la + 1 < ^ < then for n,m £ I,, 

Paif{B-\Q + m + uTa)) = e2"^("'«°'+"'^°')Poi/(B-iQ), 
where Z © Zr^ = . 

Proof. (l)Letaj e A^. Since A^ is isomorphic to SaF^So:^ there exists 7 e satisfying _Bq7_B^^(3 = 
Q+tjforallQ e (See lemma igllll). Since/ e ^(F,x), it follows that F„//(7Q) = Paix{l)f{Q)- 
However, 

PclXh) - {xilTPllT = {x{l-')Pair = {PalT = Pal- 

Thus, 

Palf{B-\Q + ^ Palf{B-'Ba-fB-\Q)) = Palf{jB-'Q) = Palf{B-'Q). 

(2) A direct calculation similar to (1) shows that Pc«;/(S-i(Q+m+nrc,)) = Paif{RaS2B-^Q) ^ 

PalXi.Ro.rx{SaTf{.B-^Q) = e2"("»«»'+"»S.0p^,/(B-lQ). □ 

Fourier Expansion: The vector case 

Proposition 3.3.4. Let F be cofinite, f e A(r,x) be smooth satisfying 
f{B^^{z + rj)) = O{r^)for some g N and each cusp Ca ofV, and Af = Xf with X — 1 — s'^, s ^ 0. 

(1) Ifl <l<ka, then 

Paif{B-\z + rj))=aaLor^+' + baLor^-' + «ai,M^ifs(27r|/i|r)e2"<^'^>. 

(2) Ifka + l<l<la, then 

PaJ{B-\z + rj))= aai,,rKsi2n\fi\r)e^^'<^^''l 

(3) If la + i < I < n, 0r^i,9s^i are chosen so that exp{2ni9 j^^^i) ~ Xr^i, and exp{2ni9s^i) ~ Xs^i, 
then 

Paif{B-\z + rj))= Y aal^^rK,{27^\^^'\r)e'-<^''^). 
where ^' = 9npi + (?sM2 + A^ is the lattice dual to A^ generated by ^1, ^2, where 

(^il, 1) = 1, {^ll,Ta) = 0, (^2, 1) = 0, {^l2,Ta) = 1, 

and the aai,ii arc scalar multiples of the vector Vai- 
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Proof. (1) By Lemina l3.3.3I F^/ fiB^^iz + rj)) is a scalar function multiplied by the vector Vai, thus 
an application of equation 1 13.3. 2b concludes the proof. 
(2) By(l) 

We will show that a^/.o and baifi are both zero. To see this first, observe that (by Lemma [3.3. Il l 

= aa/,or^+" + baifiT^^' + ©(e-l'^l'') as r ^ oo. 

Next, 

Po.if{B-\B^E^B-^){z + rj)) = a^i,or''+' + b^i.^r''-' + 0(e-l=l'-) as r ^ oo 

since BaEaB^^{z + rj) = z' + rj for some z' e Z, that is BaEaB~^ fixes the r— coordinate in 
(this follows from elementary computations and that fact that BaEaB^^oo ~ oo). However, we 
also have 

Pc.ifiB-\B^Eo.B-'){z + rj)) = P^ix{E^)f{B-\z + rj)) = 
XaiPaif{B-\z + rj)) = Xai{ao.i,or^+' + b^^or^-' + 0(e-l^l'^)), 

a contradiction since Aq; ^ 1. 

(3) Let h{Q) = h{z) = exp((/ii, z) B^ai + (/i2, 2) Osai)^ where Q = z + rj. Then h{Q) satisfies 

h{Q + m + Tau) = cxp(27ri(m6'fla/ + n9sai))h{Q), 

thus by lemma I l3.3.3t 

^^^(Q + c.)) = ^-^(g) 

n n 
for all Q e H"^ and w e A„ . Thus we can write 



which is equivalent to 

PaifoB-\Q)^ ^ g^{r)h{z)e'^^^'^'^l 

Ate AO 

To conclude the proof recall that Af — A/, / grows by at most 0{r^) near each cusp, and apply 
the separation of variables technique (see IEGM98I page 105]). □ 

3.4 The Eisenstein series 

As usual for this section let F be cofinite with cusp representatives {Ca}S=i, S~^c» ~ C,a, and 

(x, V) e Rcp(r, V) . 



3.4. THE EISENSTEIN SERIES 
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Definitions 

For P e H'', Re(s) > 1, and v £ Va we define the Eisenstein series by 

EiP,s,a,v) = EiP,s,a,v,T,x)^ J2 [r{B^MP)f+'x{Myv. 

The series E{P, s, a, v) converges uniformly and absolutely on compact subsets of {Re(s) > 1} x 
H^, and is a automorphic function that satisfies 

AE{ ■ ,s,a,v) = XE( ■ ,s,a,v). 

In addition 

EiBp'iz + rj),s,a,v) = Oir^) 

for any cusp C/3- 

Fourier expansion of the Eisenstein series 

In this section we give the explicit Fourier expansion of the Eisenstein series. 
For (^a a cusp of F and I < I < k^, set Eai = Eai[s) = E{ - ,s,a, v^i) 

Lemma 3.4.1. Let CaXfJ cusps ofr,l<l<ka, and 1 < k < hp. Then 

PpE^i{B-^\z + rj)) = 5c.pr^+'vpi + 



where 



/ 27ri</i,f> 

27ri+^ 



|r„:r^||A;3|r(i + s)' 

c = c{BaMBp^) and d — d{BaMB^^) are the bottom left and bottom right indices respectively of the 
two-by-two matrix BaMB^^. 

Proof. An application of Proposition 13 .3 .41 part (1) gives us the general form of the Fourier series 
expansion. Then apply the argument found in IIEGM98[ page 111]. □ 

A Maafi form is a smooth function / e A(r, x) satisfying f{B'^^{z + rj)) = 0{r'^) for some 
iV e N and for each cusp Ca of F, and Af = Xf with A = 1 - s^, s ^ 0. 

We saw in proposition l|3.3.41 that for 1 < ^ < fca, a Maafi form / can be expanded as 

Po.if{B-\z + rj)) = a^i^or^+-' + b^i^^r^-' + ^ a^i,^rKs{2TT\fi\r)e^^'<^^'^ . 
The 7-1+*^ T-i-i* terms will play a promanent role in the what follows. Define u/^, by 
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The scattering matrix 

Since the Eisenstein series is a Maafi form, we can apply the function u to read off its constant term 
in its Fourier expansion. Define the scattering matrix &a,i,i3,k{s) by 

We can put the scattering matrix into a fc(r, x) x k{V, x) matrix. Let 6(s) be the fc(r, x) x k{r,x) 
matrix given block-wise, 

(Sa/3(s))=(6a,i,/3,fe(s)) 

where 1 < ^ < ka, I < k < kfj. 

3.5 The Maafi-Selberg Relations 

Our main goal in this section is to prove the Maafi-Selberg relations in the case of cofinite Kleinian 
groups with finite dimensional unitary representations. The relations will be needed to prove 
the Selberg trace formula. For this section assume F is a cofinite Kleinian group and (x, V) € 

Rcp(r, V) . 
Greene's Theorem 

The basic tool for proving the Maafi-Selberg relations is Greene's Theorem. Let D c be compact 
with a piece- wise smooth boundary, /, g be complex valued smooth functions, dv the volume form 
on H^, and dv the induced volume form on the boimdary of D, dD. Then Greene's theorem is 



Our application of Greene's theorem will be to the compact part of the fundamental domain 
of r. Writing Greene's theorem in the global coordinates of using dv = ''^^"'^^"''^ - 
dv = -i- J- dv = ^^4^ and Greene's Theorem becomes, 

an ' 



dxdy 



when dD is a two-dimensional region parallel to C on the boundary of . 
The scalar Maafi-Selberg relations 

For this section, assume F has only one class of cusps and that x = 1- Let / G A{T, 1) and assume 
that / has polynomial growth as z+rj approaches the cusp at oo, is smooth, and satisfies Af = Xf 
(in other words / is a Maafi form). Let A be the lattice corresponding to F'^ . Then we can expand 

f{z + rj)= J2 5/.(r-)e'"'<'''^> 



where 

dr^ dr 
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Lemma 3.5.1. Let Af — A/, Ag = vg with A = 1 — s^, = 1 — t^, s 7^ 0, i ^ 0. Assume T has only 
one cusp at oo and 

f{P) ^ ar^+' + br^-' + ^ /^(r)e2"<''^^> 
^eAo 



Let Uf{P) = ari+" + br'^"", Ug{P) = cri+* + dr^-* 



/or h = /, ,g. T/ien 

- |P I ((« + t)bcY-'^' +{s- t)bdY-'-') 

Proof. Let D = J^o = 3^\^{Y) (here ^(y) is the cusp sector that goes to infinity). Then by Greene's 
theorem 



/ {fA-g--gA!)dv^ [ {f^g-g^f)dv. 
Jd JdD on on 



Since the (vertical) sides of D are pair-wise identified by isometrics (with opposite orientation) 
the only surviving boundary term is the horizontal cross section (the boundary where T was 
separated into JFo and T{Y)). Greene's theorem becomes 

/^(;^5_5^/,i*t=/,/35_4;,i^, (3.5.1) 

where dv = f^^^^^l^, a.^ = r-§^,dv ^ -§^^ L dv ^ and P = dD is the horizontal boundary 

region described above. 

The set P is a vertical translation of a fundamental domain of the action of Too on C. That is 
the 1 — coordinate projection is constant and equal to Y. Set 

p'^ U ip. 

Then P' is a fundamental domain for A = Aqo . 

With our notation the right side of equation B.S.lt becomes 



g — 27ri</^,z> 



dxdy 
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— ((s - f)acr''+* + {s + t)adY' 



|A| 



l-Too '■ '^oo 



— ((s + f)6cr-"+* + (s - t)bdY-''-^^ 



In the terms above involving multiplications of lattice sums, after integration over P' the only 
surviving terms are (by the orthogonality of the family {e2'^'<A'>z> j.) 



|A| 



—\Y.iu{Y)9^Y)-My).f',{Y))^ 



A direct calculation using the fact that and satisfy the Bessel equation l l3.3.Hl shows that 

-r{ ) = — (s )!An9ii(j-) 

dr r r'^ 



thus 

|A| 



riT.if^(YKiY)~-9,iY)r,{Y))^ 



Ir ■ r' I y-'t^'- i^tJ.\' I JMv- ijfj,\- II Y 



By definition /^.g^ = /.g dv and thus combing the integral of D and we obtain the 
lemma. □ 

The vector form of the Maafi-Selberg relations 

Let r be cofinite with cusps Ca ~ B^^oo for a — 1 . . . k, with fundamental domain T = J-q U 
Ti{Y)U... T^{Y). For (x, V) G Rep(r, V) . 

Let / € ^(r, x) be a Maafi form, that is / is smooth, satisfies f{B^^{z + rj)) = 0{r^) for some 
N and for each cusp of F, and Af = Xf with A = 1 - s^, s ^O.We define 

fY(p^[ f{P)~j:ti^UP) iiPeW), a = l...K 
^ \ /(F) otherwise 

Theorem 3.5.2. (Maafl-Selberg Relations) Let F be cofinite, (x, V) e Rep(F, V), f and g be Maafi forms 
satisfying A f ~ A/, Ag = vg with X = 1 — s^,i/ = \ — ,s ^ Q.t ^ Q. Suppose that u f_ai (B^ ^ P) = 
a-air'^^'' + hc.ir^-" and Ug^ai{B~^P) = Cair'^+'' + dair^^". Then 



(s'-P) I {f,g'')^dv 



^Hao.i,Co.i)yY'+' + (s + t){a^udo.i)yY'-' 



{S + t) {bo,l,Ca,l)y Y-'+' + {s-i) {b^l,dal)v Y'"'' 
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Proof. For notational simplicity we assume F has only one class of cusps at oo . We can write Poo — 
Pi + • • • + Pfc^ (see flSS- 



= {S^ - P) {{Poo + (1 - Pco))/^, (Pco + (1 - Pco))5'')^ dv 

= {S^ - P) 1^ {{Poof, Poog')y + ((1 - Poc)f, (1 - Poo)9^)y) dv. (3.5.2) 

The equality follows from the fact that Poo is an orthogonal projection. We will first show 

{s^ ~ J^(^^ - (1 - Poo)g'')y dv = (3.5.3) 

Since (1 — Poo)/ has no constant Fourier expansion coefficients (see Proposition 13.3.41 (1 — 

Poo)/"^ - (1 - Poo)/, and by LemmaHHl 

— (1 - Poo)f{B-\z + rj)) = 0(e-l=l^) (3.5.4) 

for p> and as r ^ oo. 

Next to prove equation | |3.5.3|I we apply the vector form of Greene's theorem, 

and we obtain (by l l3.5.4|l ') 

lim - P) [ ((1 - Poo)/, (1 ~ Poo).g)y dv 



= lim 

R — >oo 



(i-p.)4a-p»)/)J^^^o, 



To conclude the proof write 

k 

{S'-P)f (Poo/^,PooS^)^d.= (.^-?)E / {P,r,P,9'')ydv 

J:f ^^,J^ 
and apply Lemma l|331t to (s^ - p) (P,/^, Pj.g'^)^ dv. □ 
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3.6 The resolvent kernel 
The definition of the resolvent 

One of our main goals in this paper is to understand the spectrum of A . The spectrum is best 
understood in terms of the resolvent operator. The resolvent set of A, p{A), is the set of all z e C 
so that {A —zl) has a boimded inverse, defined on the entire Hilbert space ^.{r, x). The spectrum 
of Z\, cr{A), is the complement of the resolvent set 'C\p{A). For each z s p{A) we have the resolvent 
operator Rz = {A —zl) ^ . 



The resolvent kernel for A{T,x) 

We will understand the resolvent operator through its kernel. A bounded linear operator K : 
H^r, x) 1^ Ti-ir, x) is said to have a kernel if there exists a function k : x M'^ xV V so that for 
all / e n{r, x),P^ Kf{P) = fc(P, Q)f{Q) dv{Q). Recall (from g2J that for i > 1, s e C, 



and for P, Q e H'^, 5{P, Q) = cosh(c?(P, Q)), where d is the hyperbolic distance fimction. 

Definition 3.6.1. Let F be a cofinite Kleinian group, x G Rep(F, V), and s e C with Rc(,s) > 1. 
Define the Maafi-Selberg Series 

F^(P, Q, s) = i- ^ x{M)vs{5{P, MQ)). (3.6.1) 

Mev 

Since X^A/er Wsi^iPi ^IQ))\ converges uniformly when P, Q, and s are restricted to compact 
subsets of 

{Re(s) > 1} X (H3 X H3 \ { (P, Q) e X | FP = FQ}) (see 1EGM981 page 96]) and x is a unitary 
representation {x{l) has norm one for all 7 e F) the sum in equation (|3.6.Hl converges absolutely 
and uniformly when P, Q. and s are restricted to compact subsets of 
{Re(s) > 1} X (H3 X H3 \ { (P, Q) e H3 X H3 | FP = FQ}) . 

The function F^{P, Q, s) is the kernel for the resolvent operator of A . 

Theorem 3.6.2. Let F be cofinite with x G Rep(F, V), and A = 1 — e p(^) ^'^^ Re(s) > 1. Then 
Rx : n{r, x)^D has kernel F^{P, Q, s). 

To prove the theorem one shows that for A = 1 — with Rc(s) > 1, any element u E D can be 
represented by a continuous function which satisfies 

u{Q) = ^ ^ PxiP^ Q' ->^HP) dv{P). (3.6.2) 



Equation | |3. 6.21 1 can be seen by unfolding the integral in | |3. 6.21 1 and applying Lemma 12.2.11 See 
fEGM98. theorem 4.2.6, page 150] for more details. 
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Fourier expansion of the resolvent kernel 

In this section we will concern ourselves with a sub-sum of the resolvent kernel that causes the 
resolvent kernel to not be of Hilbert-Schmidt type. For notational simplicity we assume that F 
is cofinite with only one cusp at (a = oo. Fix a fundamental domain T = Ty U T{Y\ and let 

xeRep(r,y). 

Recall that for Rc(s) > 1 the resolvent kernel can be expressed as a sum 



4tt 
Afer 

Let 

Foo(P,g)=^ E xiM)ips{SiP,MQ)), 

and 

Fo(F, Q) = F^iP, Q, s) -F^{P, Q). 

Let {vooi} be the basis constructed in ! j3.3l For convenience we drop the oo subscript and 
consider {vi} and corresponding orthogonal projections {Pi}. 

The following lemmas are all proved using a standard separation of variables technique. See 
IIVen82l Pages 20-23] and 

Lemma 3.6.3. Let F, Q e satisfy P ^ Q. Then 



PooFoo{P,Q) = PocRoirp,rQ,s) + Poo ^ "'t^^f^i^P^'^Q^ K 

where 

^" " ^ I r]^'rl+^ rp > 

for some constant c, and 



Rf,{rp,rQ,s) 



rpKs{2^T\^J.\rp) rQls{2TT\^\rQ) rp > rq 
rpIs{2T:\^i\rp) rQKs{2T:\^i\rQ) rp < rq 



Lemma 3.6.4. Let P,Q eB.^ satisfy P ^ Q,koo + I < I < loo- Then 

PlFoo{P,Q) = 

Pi E aiMrp,rq,s)e^^^^^'''--''^\ 

where 



R / rpKs{2TT\^i\rp) rQl,{2TT\^i\rQ) rp > rq 

H^(rp,rq, s) - <^ rpW7T\,,\rp) rQK,{2^T\^i\rQ) rp < rq 
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Lemma 3.6.5. Let P,Q eB.^ satisfy P QJoo + I < I < n. Then 

PiFoo{P,Q) = 

where 

p, . , _ ( rpK,{2Tr\i/\rp)rQls{2TT\^i'\rQ) rp>rQ 

^" I rp/,(27r|M'|rp)rQif,(2^|Ai'|rQ) rp < rg ' 

and n' = Or^i + 05^2 + (^See Proposition \3.3.4\f or an explanation of the notation used). 



Estimates of the resolvent kernel 

It follows from IIEGM981 Theorem 4.5.2] that 



\Fo{P,Q)\^y dv{P) dv{Q)<(x,. 

In other words, Fq (P, Q) is a Hilbert-Schmidt kernel. On the other hand, F^a (P, Q) is not. How- 
ever, from the Fourier series expansion of Foo{P, Q), and the decay properties of the Bessel func- 
tions rIs{2Tr\fi\r), rifs(27r|/i|r) it follows that the only component of the resolvent kernel that de- 
stroys square-integrability is. 



Roo{P, Q) = -Poo \ S-sJl- 

I 'p 'q 



Definition 3.6.6. Let F be cofinite with one cusp at = oo. For B >Y decompose = J^o(JJ^{B), 

and let x G Ilep(r, V), define 

and FeiP, Q) = F^{P, Q, s) -F^iP, Q). 

We have shown the following: 
Lemma 3.6.7. 

\FBiP, Q)fv dv{P) dviQ) < oo. 



3.7 Analytic continuation of the Eisenstein Series 

The meromorphic continuation of E^i {P, s) is necessary for the proof of the spectral decomposi- 
tion theorem, and is highly non-trivial. Fortimately, there are several well-known methods avail- 
able, |IFad691, fSei89l, fSeMI, rCdVSl], and |'Lan761. In fEGM98*| an adaptation of the methods in 
[CdV 81] is used to prove the three-dimensional scalar case, and a similar adaptation works^ for 
the vector case which we show now. 



^Alternatively, Faddeev's method can also be used to express the meromorphic continuation of Eci{P, s) by adapting 
|Ven82| Chapters 2 and 3]. 
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The idea of the proof 

As in Faddeev's proof the key idea is the resolvent kernel of A . We know from i]3.6l that it is not a 
Hilbert-Schmidt operator. However the key step in the proof is to construct a self adjoint operator 
Aa defined on a closed subspace (the set of functions whose constant Fourier term vanishes for 
r > a ) so that {Aa — A)^^ is Hilbert-Schmidt and is closely related to A . 

Statement of the theorem 

Theorem 3.7.1. Let T be cofinite, (x, V) € Rep(r, V), and 1 < I < ka- Then 

(1) Eai{P,s) admits a meromorphic continuation in the following sense. For each fixed P e H'^, 
Eai (s, P) is a meromorphic function in s e C. The poles of Eai {P, s) depend only on s and not on P. 

(2) IfU c C is open with Eai{P, s) regular on U then Eai is real analytic onU x H-^. 

(3) For each regular s e C,AEai{P, s) = (1 - s^)Eai{P, s). 

(4) The scattering matrix & {s) and Eai{P, s) are both finite ordered meromorphic functions with order 
< 4. 

(5) £{P,~s) = &{-s)£{P,s) 

(6) e{s)e{-s) = 1. 

(7) & {s) is a unitary matrix on the critical axis Re(s) = 0. 
Proof of theorem (|3.7.1|) part I: Preliminaries 

In order to avoid complicated notation (quadruple subscripts) we will assume the following: 

Assumption 3.7.2. The Kleinian group T has only one class of cusps at ( ^ oo <eP, and x G Rep(r, V) . 

Then we can choose fundamental domain of the form !F = TqU J-{Y) (here J-q = Ty , see i)2.6l 
). We can write the orthogonal projection onto V^oo by 

Poo =2^'=' 

k=\ 

where P^ is the orthogonal projection onto V]. — v^ok (see i l3.3|l . 

Since we assume that oo is the only cusp we have (by definition) /cqo distinct Eisenstein series ( 
Eooi {P,s), / = 1 . . . /coo ) which have an expansion of the form (see ^3.31 ^3A\i . 

PkEooiiP, s) = Skir^+'vk + M^y^'^k + 0(e-l^l'-). (3.7.1) 

We will need various spaces of functions. Let irr : H'^ ^ F \ H'^ be the standard (quotient map) 
projection, and set 

V°° ={$ e A{T, x) n C°°(H3, V) 1 7rr(supp($)) is compact in F \ H^}. 

We will restrict functions in T>°° to The space of distributions V is the set of all continuous 
(complex valued) linear functionals on A function / G Lj^^(r, x) can be identified with the 
distribution U where 

[/($) ^ ^ mP)JiP))y dv{P). (3.7.2) 
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Noting that H{r, x) C Llod^^ ^^Y ^^^^ ^ distribution U eTi.{r, x) if and only if there exists 

/ e n{r, x) so that equation holds for all $ e 1?°°. 

For Y chosen in the decomposition !F = TqU J-{Y) define 

and oo) the corresponding space of distributions (the space of continuous, complex valued, 
linear functionals of oo) ). For each h E ^10^(^1 °°) associate the distribution 

hi^) = <^{r)h{r) - i^eViY,^). 

For L/ e I?', $ e i e N U {0}, we define the following differential operators at each point 

{x,y,r) e : 

Z\C/=(42]+^[2]+^[2]_^[l])[/. 

An unraveling of the definitions shows that A C/($) = U{A $). For / e TL{r, x) we abuse notation 
and define 

grad(/) e nm x) iff 4"/, D\'\f e W(r, x) . 

Let f,ge U{r, x) with grad(/), grad(g) e x) • Then we define 

An application of Greene's theorem shows that for $ e , 

Qif,^)^ l^{A<^>J)y d« = (Z\ /)($), 

where Z\ / is taken in the distributional sense. 

We next define useful maps between the function spaces 'D°° .ViY, 00) and the distribution 
spaces V ,'D'{Y, 00). In order for the notation to be simple, we will use uppercase Greek letters 
for functions in 2?°°, uppercase Roman letters for distributions in V, lower case Greek letters for 
functions in 'D{Y, 00), and lower case Roman letters for distributions in ^'{Y, 00). It is also useful 
to think of 'D°°,'D' as BIG spaces and V{Y, 00), ViY, 00) as small spaces. 

For F e H{r, x), 1 < ' < fcoo set 

Li[F]{r)=-^ / {PiF{x,y,r),vi)y dxdy. 
W Ja\C 

By Fubini's theorem Li[F] e r~^dr). For any function 

(j) : (Y,oo) C, 
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and 1 < Z < fcoo set 

b{r)vi if P = z + rj e Too{Y) 



BimP)-: 



if P e J^o 



We will identify L;[0](P) with its automorphic extension in A(r, x). 
For each / e 1 . . . fcoo define : D' D'(Y, oo) by 

Bi.4Um = U{Bi[cf>]), 

where [/ e P', e V(Y, oo), and define L'^V'(Y, oo) V by 

where u G ^'(y, oo), <t>eV°°. 

Define Z\o X>'(r, oo) ^ V'{Y, oo) by 

(Z\o u){(b) = u( (-r^^ + r4-) A (u G V (Y, oo)), € (X)). 



c?r^ dr 

Then it follows that for u e ^'(r, oo), [/ e P', Z G 1 . . . koo, 

L*i {Ao u) = A L\ {u) and 5;, (Z\ U) = Aa B^ (U) 

For a > y, s e C set 



r < a 



and T"" = Lj* [(5a] , where 5a G P'(y, oo) is the Dirac delta-distribution at the point a. 

A simple calculation using Greene's function techniques for second order differential equations 
(see Hwa02[ Pages 194-195] for details on the technique) shows that 

(Ao-il - s^))7^a,s = -2sa'-'Sa, 

and an application of L;* (/ € 1 . . . /coo) gives us 

(A -{I - s^j)Bi['na,s] = -2sa'-^Ti^. (3.7.3) 

Proof of theorem (|3.7.1|) part II: Lemmas 
Lemma 3.7.3. Let 

={F e W(r, x) \Li[F]{r) =0,r>a,l el... fcoo}, 
I),={Fei3a|grad(F)eH(r,x)}, 

and for F,G eDa 

QaiF,G) = Q{F,G). 

Then 

(1) Sja is a closed subspace ofH{r, x) • 

(2) There exists a self-adjoint operator Aa with domain T^a^ C Sa C iOa with 

Aa ■ ^ S^a 



34 



CHAPTER 3. THE SPECTRAL DECOMPOSITION THEOREM 



that satisfies 

J {AaF,G)y dv^Qa{F,G) 

for all F e Da^,G£ Da, 

(3) consists of those F e Da which satisfy 

fcoo 

AF = G + ^ciT[', (3.7.4) 

1=1 

for some G € Sja- If equation (|3.7.4|l is satisfied, then AaF = G. 

The proof (of the above lemma) is similar to IIEGM981 Pages 237-239]. 

Lemma 3.7.4. (2jFo/'Re(s) > 1, [Aa —(1 — s^))^^ is a Hilbert-Schmidt operator. 

(2) The self-adjoint operator Aa has a purely discrete spectrum. 

(3) The resolvent {Aa —(1 — s^))^^ can he continued to an operator valued meromorphic function on 
all ofC of order < 4. 

Proof (1) and (2) follow from a straight forward adaptation 1EGM98[ Prop. 6.1.8, page 240-241]. 
(2) follows from standard functional analysis while (3) is proved in 1EGM98I Lemma 6.1.9, page 
240]. □ 

Let hs{r) = h{r)r^+' where for Y < Y' < Y" h : (Y, oo) ^ R smooth and h{y) = 1 for y > Y" 
and h{y) = for y < F' . 

LetHi{P,s) = ((zi-(l-.s2))B([/i^](P).ThenitisinH(r,x) and smooth. In fact it is alsoinija- 
Let 

17={s e C I Re(s) > 0, 1 - ^ cr{A)}. 

Lemma 3.7.5. Let Y < Y' < Y" and h : {Y,oo) Rbe a smooth function satisfying h{r) — \ for 
r > Y" and h{r) — Ofor r < Y' . For each / e 1 . . .k^o set 

Hi{P,s)^{{A-{l-s'))Bi[hs]{P), 

and for s £ flset 

E\P,.s)^Bi%]{P) - {{A -{I - s')r'HiiP,s). 

Then 

(1) Bi[hs] e A(r, X) is a smooth. 

(2) Hi{z + jr, s) = Q for all r > Y" and hence is in H{r, x) ■ Moreover, ifY" < a then Hi{-,s) £ Sja- 

(3) The function Hi{P, s) is holomorphic for s £ C. 

(4) The function E\P, s) is holomorphic for s £^l. 

(5) The function E\P, s) is the unique function in A{T, x) that satisfies 

{A-{l-s^))E\P,s)=Q, E\P,s)-Bi[K]{P)£n{r,x)- 

(6) For all P e H^, s £ fl, E\P, s) = E^i{P, s). 
See IIEGM98[ Pages 234-235]. 
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Proof of theorem (|3.7.H) part III: The meromorphic continuation 

We exhibit the meromorphic continuation of the Eisenstein series to all of C. 
For s eC with 1 - ^ fia = (T{Aa) set 

By Lemma r3.7.4| and Lemma r3.7.5[ Ai (P, s) varies meromorphically. Next for each s e 57^ A; (P, s) £ 
Da^ (since it is in the range of the resolvent of Aa), and an appeal to Lemma [3.7.4| give us 

{A-{1 - s'))Ai{P,s) = m{P,s) + Y,ciA''^)T^- (3-7.5) 

Plugging in (to the distributional equation above) arbitrary test fimctions $ G which do 

not vanish at the support of each T", into equation l|3.7.5|l shows that the functions ci,j{s) are 
meromorphic on all of C (the remaining terms in equation I l3.7.5b are meromorphic). 
Next define the following meromorphic functions, 

e/j(s) = ^'S"^a^~"cij(s). 

Finally, let 

Ni (P, s) = Ai (P, s) + eij is)Bi [Va,s] {P) + Bi (P). 

Then (in the distributional sense) {A —(1 — s'^))Ni{P, s) = 0, and moreover, by the elliptic regularity 
theorem Ni{P,s) is smooth as a function of P G . 

Since ei,j{s), Bi[hs]{P) are meromorphic, and of order < 4, Ni{P,s) is meromorphic, and of 
order < 4. 

To relate Ni{P, s) to Eaoi{P, s), express EaoiiP, s) as a linear combination of the Ni{P, s) and 
apply the uniqueness part of Lemma [3. 7.51 

We have proved parts (l)-(4) of Theorem 13.7.11 Part (5) can be seen by plugging in — s into 
Eooi {P, s) and applying the uniqueness part of Lemma 13.7.51 Part(6) follows immediately from 
(5). Part (7): from the Dirichlet series representation of the components of 6(s) we see that for 
Re(s) > 1, 

6(s) = 6*(s). 

Observing that for s — it,t eR, it — —it and appealing to (6) proves (7). 
3.8 Proof of the spectral decomposition theorem 

There are several proofs of the spectral decomposition theorem. All of them use the analytic con- 
tinuation of the Eisenstein series to the critical line Rc(s) = in a crucial manner. 

One particularly nice approach is to use the general theory of eigenpackets. This approach 
makes the deep connection between the Eisenstein series and the spectral decomposition theo- 
rem quite clear. We shall see that integral of the Eisenstein series on segments of the critical line 
(Rc(s) = 0) is an eigenpacket. 
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Statement of the theorem 

Let r be a cofinite group with (x, V) G Rep(r, V) . 

Let D denote a countable indexing set of all u e D (the domain of A) such that Au ^ Xu for 
some A e C. 

We remark that in general, we do not know if D is an infinite set. 
Theorem 3.8.1. Let f £ D. Then f has an expansion of the form 



The sum and integrals converge pointwise absolutely and uniformly on compact subsets o/H^ . 
Eigenpackets 

Throughout this section, let A : Va ^ H be a self-adjoint operator with domain Va in the Hilbert 
space H, with inner product <•,•>. 

We will outline the spectral decomposition theorem for eigenpackets. See | EGM98[ Section 6.2] 
for more details. 

An eigenpacket of A is a map w : M ^ H, A i-^ having the following properties: 

(1) Wo = and vx G Va for all A e M. 

(2) The map u : R ^ H is continuous (in the norm sense). 

(3) Av\ = fj, dv{p), where the integral is the limit in the norm sense of the corresponding 

Stieltjes sums. By an integration by parts, we can rewrite (3) as Av\ = Xvx — dp,, where the 

integral is the limit of Riemann sums in the norm sense. 

Eigenpackets have many similarities to eigertfimctions. If / is an eigenvector of A then < 
/, v\ >= for all A e R. In addition, if w is also an eigenpacket of A and [a, f3], [7, S] are two 
intervals having at most one point in common, then 



Fix an eigenpacket v oi A. Then there exists a non-decreasing right continuous fimction F : 
R ^ M which is unique up to an additive constant satisfying the equation 



f{P) 




< f,E^i{-,it) > E^i{P,it)dt, (3.8.1) 



< Vp — Va,WS — W-y >— 0. 



(3.8.2) 



F{f3)-F{a) ^ \\vp-va\ 



(3.8.3) 



for all a < /3. 

For X e H define a function 1^9 : R ^ C, by 



tp{X) =< x,vx > ■ 



(3.8.4) 



Next let T be a partion of [a,f3], of the form 



T : a = Ao < Ai < • • • < A„ = /3, 
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and define 

For a sequence of partitions (sub partitions) T„ with maximal width of its subdivsion converging 
to zero set 

'■'^ dipiX) dv{X) 



dF{X) 



= lim xt„. 



Further define 



We can now write down the general spectral decomposition theorem. Let D be a countable 
indexing set for a maximal set of orthonormal eigenvectors A, and 91 a countable indexing set for 
a maximal set of orthogonal eigenpackets of A. 

Lemma 3.8.2. Let {ejn}mev be a maximal set of orthonormal eigenvectors of A, and {vn}nem a maximal 
set of orthogonal eigenpackets of A. Then every x e H can he decomposed as 



dF„{X) ■ 



mfci; nfc-jc 

The sum and integrals converge in the norm sense in H. 
Proof of the theorem 

Lemma 3.8.3. Let T be cofinite with x G Rep(r, V) . For P e H^, a e I. . .k, 

I £ 1 . . .ka,we define 

forX<l 



V^\P)^ 



E^i{P,it) dt forX>l 



10 

with positive choice of square root. Then (•) G H(r, x) ^nd the family o/F["'1 are a maximal system 
of orthogonal eigenpackets for A . Further, 

\\vif - Viff = 27r^-^(Ti - T2) (3.8.5) 

for Ai > A2 > 1 and Ti = VAi - 1, T2 = VA2 - 1. 

Proof. For notational simplicity we work under Assumption l3.7.2l From the Fourier expansion of 
the Eisenstein series we see that E^i{P, s) = s) + ©(e^l"^''") as r — > 00, where Ai{P, s) — 

r^+'^vi + (l)i{s)r'^^'', and 0/(s) is defined by < (j)i{s),Vk >v^ &{s)Lk = &{s)ooi,ook- Since S(t) is 
unitary on the critical axis, \4>i{it)\v = 1 for t G M. 

We next show that V^l'' = V^l"'' S HiP, x) for each A e M. Put T = VA - 1, and set 

(3i{P,T)= [ Ai{P,it)dt. 
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An integration by parts shows that^ 

A(P,T) = 0(-^) asr^oo. 
log r 

Hence Pi{P, T) e L'^{T). The upper bound on /3(P, T) and the dominated convergence theorem 
imply that V^"^ is continuous in the norm sense. 
Next, for A > 1, 



AVI\P)^ f ^ \l + t^)E^iiP,it) 
Jo 



dt 



The last equality was obtained by subsituting /i = 1 + We have now shown that vj[^ is an 
eigenpacket. 

We next prove the orthogonality relation 

^i^^f ) = Ofor/ 7^ fc, 
using the Maafi-Selberg relations (Theorem |3.5.2| |. We have 



^i" , Vx^ ) = , lini . ( / Ef (P, it) dt, I El {P, It') dt 



= lim 



T rT 



Y^oo |Aoo| JO JO 
T i-T 



{EY{P,it),El{P,it')) dtdt' 



Jo 



lim 



t + t' 



t + t' 



t' ~t 



dt dt' = 0, 



since all three terms above are bounded for t, t' E [0, T] {&{it) is a unitary matrix so its terms are 
bounded and rows are orthonormal), and by the Riemann-Lebesgue lemma. 

Equation ||3. 8.51 1 follows in a similar manner. Letting Ai = 1 + Tf, A2 = 1 + T|, we have 



II Ai A2II Y^oc 
|Aa| 



T2 JT2 



{E[{P,it),El{P,it')) dtdt' 



Ti /.Ti 



t + f 



t + t' 



t' - t 



t' -t 



|Ao| 



dtdt' ' . m-Ta) 

[-La • -L ctj 



^The implicit constant depends on T. 
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See IIEGM98[ Pages 272-275] and IIwa021 Page 97] for the details. The trick is to add and subtract 
the conjugate to the second to last term (on the left side of the equal sign) and combine one of the 
extra terms with the last term. 

Finally, it remains to prove completeness. Let A > Y, and extend the function 



to Hir, x) ■ Then one can verify that f j is orthogonal to the set of cusp forms^ , and to all eigen- 
packets V^'^l for k ^ I. The function is not orthogonal to the exceptional eigenfimctions (the 
residues of the Eisenstein series Ei{P, s) at the poles aj e (0, 1] ) and the eigenpacket Let Tj 
denote the orthogonal projection onto the space spanned by the exceptional eigenfunctions and 
the the span of {Va^ — vj''}a,/3gR. Then a computation shows that — || that is our 

system eigenfunctions-eigenpacket system is complete with respect to expanding . Now sup- 
pose that there exists an eigenpacket W orthogonal to all of the I'l }i=i...k^ , and cusp forms, and 
let U be the span of {Wa — W/jja^^gR. Then for u e U by the completeness with respect to /](', 



ri\u) = Ofor; = l...fcoo,that is u is a cusp form, a contradiction unless u = Q. See IIEGM98I 




vi for P e J'iA) 
else 




Pages 274-276] for more details. 



□ 



*The space of cusp forms is the space spanned by the eigenvalues of A that are in ^{{r, x), and whose constant term 
in their Fourier expansion is identically zero. Only finitely many eigenvalues of A are not cusp forms. 



Chapter 4 

The Selberg Trace Formula 



In this chapter we derive the Selberg Trace Formula for cofinite Kleinian groups with finite dimen- 
sional unitary representations. 



4.1 The Selberg Trace Formula 

Theorem 4.1.1. (Selberg Trace Formula) Let T be a cofinite Kleinian group, x G R-ep(r,y), h be a 
holomorphic function on {s e C| |Ini(s)| < 2+5} for some 5 > 0, satisfying h{l+z^) — 0(l + |zp)'^/^^'^) 
as \z\ — » oo, and let 

9i^) = 77-/^^(1 + i^)e"**'^ dt. 

Then 

vol (r \ h3) 



^ vol r\M^ r , 

y /i A™ = V4 dime V h{l + t^)f dt 



^ ti:vx{R)9{Q)\ogN{To) ^ try x{T)9{\ogN{T)) , . . 



tr(6(0))/i(l) 1 



h{l + t^)--{it)dt 



4 47r 

|C(5a/c)| V Jo coshx-l + aafe 

K dime V 

+ En^^HM E (4.1.1) 

Here, {Xm}mev are the eigenvalues of A coimted with multiplicity. Following IEGM98I sec- 
tion 5.2, the summation with respect to {-Rjnce extends over the finitely many F— conjugacy classes 
of the non cuspidal elliptic elements (elliptic elements that do not fix a cusp) i? G F, and for such 
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a class N{To) is the minimal norm of a hyperbolic or loxodromic element of the centralizer C (i?) . 
The element R is understood to be a fc— th power of a primitive non cuspidal elliptic element 
Rq G C{R) describing a hyperbolic rotation around the fixed axis of R with minimal rotation an- 
gle j^^j^-^ ■ Further, £{R) is the maximal finite subgroup contained in C{R). The summation with 
respect to {T}iox extends over the T— conjugacy classes of h5^erbolic or loxodromic elements of F, 
To denotes a primitive hyperbolic or loxodromic element associated with T. The element T is con- 
jugate in PSL(2, C) to the transformation described by the diagonal matrix with diagonal entries 
a{T),a{T)-^ with \a{T)\ > 1, and N{T) = \a{T)\^. For s e C, 6(s) is a k{T,x) x fc(F,x) matrix- 
valued meromorphic function, called the scattering matrix of A, and 0(s) = det &{s). Furthermore 
Cakidaki ^ud d^k are constants depending on F which will be determined in the case of F having 
only one cusp at oo. The remaining notation will be defined in this chapter^. 
For P = z + rj, P' = z' + r'j e set 

^ ' ' 2rr' 

It follows that (5(P, P') = cosli((i(P, P')), where d denotes the hyperbolic distance in H'^ . Next, for 
k S iS([l,(X))) a Schwartz-class function, define 

K{P, Q) = k{5{P, Q)), and Kr{P, Q) = J2 xh)K{P, iQ). 

The series above converges absolutely and uniformly on compact subsets of H'^ x H^, and is 
the kernel of a bounded operator /C : T-L{r, x) ^ ^(^, x) ■ The Selberg trace formula is essentially^ 
the trace of JC evaluated in two different ways: the first using spectral theory, and the second as an 
explicit integral. 

The function h that appears in the Selberg trace formula is the Selberg-Harish-Chandra trans- 
form^ of fc, defined as follows: 

MA) = Ml-").f/°°*(i(<4))«--<-)(<-i)f. A=l-.^. (4.1.2) 
Expansion of Ky 

We now diagonalize K,. For v,w E V define a map v : V ^ Vhy 

V ^w{x) —< x,w > V. (4.1.3) 

An immediate application of the spectral decomposition theorem and the Selberg transform give 
us, 

^Please note that there is a typographical error in the loxodromic and non cuspidal elliptic terms in IEGM98I Theorem 
6.5.1; both terms are missing a factor of J- . 

^We say "essentially" because K is not of trace class. Selberg's procedure is used to define and compute the regularized 
trace. 

^If / : I— > y is a smooth function satisfying A f = \f, then Kf = h{X)f. That is / is an eigenfunction of /C with an 
eigenvalue that depends only on A. 
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Lemma 4.1.2. Let k G S and h : C ^ Che the Selberg Transform ofk. Then 

Kr{P,Q) = ^ h{Xm)em{P) <^^M 

me'D 

h k 

+ ^ E E ^nXT^ [ ^(^+ ^E„iiQ,it)dt. (4.1.4) 

T/ze swm and integrals converge absolutely and uniformly on compact subsets o/H^ x iP. 
Set ^ ^ 

^'^ a=i ;=i 

rnel' 

Then clearly ifr = I/r + -ffr- We have 
Lemma 4.1.3. 

\\Lr{P, Q)fydv{P) dv{Q) < oo, 



/ tvv{LT{P,P))dv{P) = h{Xm)and, 

\h{Xm)\ < OO. 



me'D 

Proof. The first equation follows from the decay properties of h, the orthonormality of the and 
that fact that 

|2 



/ \\Lr{P,Q)fvdv{P)dv{Q) < C J2 \H^m)\' 
For the second, note that 



< oo. 



try(e„(P) (g) em{P)) = {em{P), em{P))v 
thus by the orthonormality of the Sm, 

I tVv{LT{P,P))dv{P) = J2 K^m) f {em{P),em{P))vdv{P) = J2 K^m). 

By the decay properties of h, 

Y \h{Xm)\ < 00 (4.1.5) 

me'D 

□ 

If we naively try to take the trace of /C by Jj^tiviKriP, P)) dv{P) we would see that the 
integral does not converge. However we can regularize the trace by subtracting off the term 
iT^v {Ht{P, P)) dv{P) in the following careful maimer. 
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Lemma 4.1.4. Let f{P) = iiv{Kr{P, P)) - Itv{Ht{P, P))- Then f is in L^{T). Further, 
lim / irv{KT{P,P))dv{P)- Mm / trv{Hr{P, P)) dv{P) 



j^f{P)dv{P). (4.1.6) 



(The set Ta is defined in Lemma \2.6.4\ ) In particular, the singularities arising from each integral must 
cancel out. 

The fact that the singularities arising from each integral must cancel out will be crucial in our 
derivation of the Selberg trace formula, and imply some interesting group relations of F. 

Proof of the Spectral Truncated Trace 

For this section Assumption 13 . 7.2l is in effect. So far we know little about the t— dependence of the 
Eisenstein series on the critical line. The following consequence of Lemma 14.1.21 will remedy the 
situation. 

Lemma 4.1.5. Lef / e 1 . . . fcoo ■ Then 

i-T 



f \Ei{z+jr,tt)\ldt = 0{r^T + T''). 

J-T 

Proof. For < e < 1/16 let fee : [l,oo) Mbea smooth function satisfying 

supp(/c,) C [1,1 + e], max(|fc,|) < ce-i, / k^{5{P,Q)) dv{Q) = I, 

for some c > 0, and for all P e IH'^ (See IEGM98i page 292] for the construction). Then it follows 
( IIEGM98I page 292]) that 

\\Kr{P, Q)\\l dv{Q) < ci (r%e-i + £-§) 
for all P ~ {zp, rp) £ T . On the other hand we have (Lemma l4.1.2|l 

V /i(A™)e„(P) ® era{Q) + 1^ C^oo / h{l+t^)Ei{P,it)®EiiQ,it)dt, 

rneV 1=1 

and by Parseval's equality (for the Hilbert space 7Y(-r, x))> setting e = (16r^)^^, we obtain 

ciirlT + T') > WKriP, > ^ IMl + t')miP, zt)P dt. 

By IIEGM981 page 122] we get 

IMl + 0-l|<^(l + l<l)^, for|t|<T, 
which gives a positive lower boimd on \h(l + for t E [— T, T]. The lemma now follows. □ 
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The following lemma will allow us to compute the spectral trace. More specifically the Maafi- 
Selberg relations will give the leading terms of the truncated trace while the error term (below) 
will vanish as A oo. We adapt a nice argument found in Iilwa02t Page 142] which shortens the 
exposition found in IEGM98[ Page 295]. 



Lemma 4.1.6. Let h{l + z^) = 0((1 + |z|2)-3/2-e). Then 

h{l + t^)\Ef{P, it)\y dv{P) dt = 0{A-^) asA^oo. 

r{A) 



Proof. Let Coo = |'^' p and let T-" be a fundamental domain for the lattice Aqo. By Parseval's 
equality* (for eigenf unctions of the euclidean Laplacian on C — M'^),) 



1 



i?/4(z + n,ii)|2,dz= V|0,,^(ji)|2.|rif,,(2^|Ai|r)|2. (4.1.7) 



We have the well known formula 



which implies (by an elementary argument) that for any < n < m, constant A, sufficiently large, 
and implied constant independent of t, 



/"OO /.oo 

/ \K,t{r)\\-'' dr « / |X,t(r)|2r-™ dr. (4.1.8) 

J A J A/2 

An application of Parseval's equality, and | |4. 1.71 1 gives 



is the Fourier coefficient of Ei{P, s). 
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( \Ef{P,zt)\l dv{P) 
Jr{A) 



IP .r'] I I \Ef{z + rj,zt)\l.dz^ 

[i oo ■ i ooj J A J-P ^ 
1 J 
7^/ ^|0.4^^)Py|ri^.t(27^|M|r)p- 



^|0z,^(zt)|2,|if.,(27r|/i|r)|2^ 

ja/2 ~rn 



A/2 l^oo 



- / \E^'\z + rj,zt%dz- 
I J-p ^ 



< 



A/2 l^oo 



- / \Ei{z + rj,it)\y dz -j 
I Jv 



Hence, we infer from Lemma 14.1.51 and another application of Parsavel's equality (for eigenf unc- 
tions of the Euclidean Laplacian) that 



/ / \E^{P,it)\ldv{P)dt 

J-T Jr(A) 

« f r I \Et'^z + rJ,^t)\ldz% 

J -T J A/2 Jv 



I -T J A/2 JV 

i-oo t-T 



f f dr 

<C / \Ei(P,zt)\ldt- 

Ja/2J~T ^ 



The lemma follows from the bound on h{l + z^) using an elementary integration by parts argu- 
ment. □ 

Define (f>{s) = det & {s). By Theorem \3.7.1\ d>(s) is a meromorphic, finite ordered function on C, 
|(/)(s)| = lon{Re(s) = 0}, and ^(s) is regular on {Re(s) = 0}. Let + z^) = 0{{l + \z\^y^/^-^), 
and 

9ix)^^J^hil + t')e'''^dt. 

Then we have 
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Lemma 4.1.7. 



/ tTv{Hr{P,P))dv{P) 



1 f , .2^.. . /i(l)tr6(0) 



The integral converges absolutely. 
Proof. 



ff(0)fc(r, x) HA) - — / ^{it)hil + t') dt + ' \ ' ' + 0(1) . (4.1.9) 



47r / tYv{Hr{P,P))dv{P) 
J:Fa 

= Y1 1 / h{l + t^)Ei {P, it) (g) El {P, it) dt dv{P) 

= 1 h{\^t')Y^ ( irCoo\Ei{P,it)\l dv{P) dt 

h 

= h{l + t^)Y, trC^\E^{P,it)\ldv{P)dt 

= [ h{i+t^)y] [ iTCoo\Ef(p,it)\idv{p)dt 

- [ h{l+t^)y] [ trCoo\E{'{P,it)\l dv{P) dt. 

By Lemma [4.1.6l the term on the last line tends to zero as A ^ oo. We evaluate the first term above 
with the Maafi-Selberg relations on the critical line Re(s) = 0. For t e R, i' — t + r, r > 0, Coo — 
^'"|A''"r^ '^^(^1 *'•) the column vector of Eisenstein series {1 = 1... koo). Consider the k^o x ^oo 
matrix^, {C^£^ {P,it),£^ {P,it'y) . Then 

k 

tr {CaoS^iP, it), S'^iP, It' )') f iiCoo\Ef{P,it)\ldv{P). 



Applying the Maafi-Selberg relations fcoo x k^o many times and placing the result in matrix form 
gives us^ 

(Coof^(P,zi),f^(F,z<')*) = 



^Each entry is the inner product of two truncated Eisenstein series. The inner product takes place in the Hilbert space 

H{r,x)- 

''We are only interested in the trace of the matrix {CooS^ (P, it), £^ (P, it'Y) . However by working with the entire 
matrix we can use the identity )4.1.10t and derive a nice expression for the trace. 



48 



CHAPTER 4. THE SELBERG TRACE FORMULA 



t + t' t + t' t' -t t' -t ' 

Next applying the approximations, 

F*'' = i + ?:riny + ..., 

y^*'' = I - irlnY + . . . , 
6{it + ir) = G{it) + ir&{it)' + 

identities, 

6{-it) = &*{it) = &~^(it), 

^e{it)e{~-it) = -i&{it)&'{-it) + i&' {it)e{-ii) = = o, 

tr6'(it)6-^(it) = -^logdetefii) = ^(it), (4.1.10) 
at (j) 

and letting r ^ we obtain the lemma. See IEGM98I Page 305] and IIwa02[ Page 139-142] for 
more details. □ 

4.2 Lattice Characters and Sums 

As mentioned earlier the parabolic elements of a cofinite Kleinian group have an associated lattice. 

Z(x,A,V) 

Let A = Z0ZrcCbea lattice with Ini(T) > 0. A (lattice) character ■0 of A is a one-dimensional 
unitary representation of A. 

Definition 4.2.1. For a; > set 

MSA 

Ia'I'<^ 

and when the limit exists 

i:(A, '(/')= lim A,i/'). 

X — >oo 

Proposition 4.2.2. Let ipbe a character of K. 

(1) Iftp — id, the trivial character, then 

Z(a;,A,'0) = |^(lna:: + ?7A) + 0(^a;~^^ fls x -> oo. (4.2.1) 

(2) If t/i 7^ id then lim^^oo Z{x, A, -0) exists and 

Z(a;, A,i/') = ^lA,-!/") + O ^a;"^^ as a; oo. 
Here r\(^ can be thought of as an analogue of the Euler constant 7 for the lattice A C . 
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Proof. (1) This is proven in |EGM98| page 298], 
(2) By Lemma|123] 

a:<|p|2<p 

Here the impUed constant does not depend on p. Hence, convergence follows from the Cauchy 
criterion. Letting p oo shows that 



□ 



Kronecker's Second Limit Formula 



Let u, V be real numbers which are both not integers and write t = x + iy {y > 0). For Re(s) > 1 
set 

i?,,,(T,s)= lim V' e2-(""+™)^ ^1^. 

x^co ^ — ' IrriT + nr^ 

\mT+n\^<x ' ' 

Here the prime in the sum means to leave out zero. The series converges uniformly and absolutely 
on compact subsets of Re(s) > 1. Thus we can also define 



IrriT + nP** 

(Ji,m)^(0,0) ' ' 

since the order of summation is not important when a sum converges absolutely In the sums, n 
and m are understood to vary over the integers. 
We have ( ISiegOll or ILan87[ page 276]^ 

Lemma 4.2.3. The function Eu,v{t, s) can he continued to an entire function of s e C, and one has 

1) = -27rlog|g_,,„(T)|, (4.2.2) 

where gai.as the Siegel function, 

oo 
n=l 

B2iX) = - X + l/6,qr = e^"^, = e^^^^^ and z = aiT + az. 

We now explain the relationship between L{A, tp) and Euai (t, !)• The character ip is determined 
by u and v. That is u and v can be chosen so that ■0(1) = e'^^™ and iP{t) ~ g'^^™ . We now can rewrite 

Z{x,A,ij)= 1 —1^- (4.2.4) 

\mT-\-n\^ <x 



^There appears to be a typographical error in ttie definition of the Siegel function on page 276 of the second edition of 
ILan87l . The correct definition appears on page 262. 
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Here the prime means we leave off zero from the sum. 

Formally, ignoring convergence and order of summation, 

g27ri(mit-f Til?) -j^ 

limZ(x,A»= y ■ — — = -Eu,v{t,1) 

»oo -"^ — ' TOT + nr V 

In the next section we will make the above argument rigorous. 
Evaluation of L{A, ifj) 

Proposition 4.2.4. Let A~Z(BZTClCbea lattice with Ini(r) > 0, a character of A, and u,v € R 
are both not integers satisfying ipil) — e^^™ and iP{t) = e^"^™. Then 

L(A,V') = — log|5-.,„(r)|. (4.2.5) 

y 

Proof. For Re(s) > 1 recall that 



y 'Eu,v{t,s) = lim y 

\mT+n\^<x ' ' 



For t e [1,2] set 

/w^iim y' 



x^oo '■ — ' Imr + np* 

|mr+n|2<a; ' ' 



By proposition ||4.2.2| | liniaj^oo Z{x, A, i/") — L{A, ip) converges, and by equation (|4.2.41 L(A, ip) = 
/(I). By Lemma |4.2.3| i/~''ij^„ „ (t. s) can be continued to an entire function (also denotedby ?/~*i?„^t,(r, s)). 
In particular y^'' Eu^v{t, s) is continuous and 

— 27r 

lim y^*Eu,v{T,t) ^ y^^E^.vir,!) = log (r)| . 

t^i+ y 

By definition, f{t) = y^*" Eu^v{t, t) for t E (1, 2]. If we can show that / is continuous on [1, 2] then 



L(A,V)-/(1)- lim lim y-'E^^,{T,t) = y-^Eu,,{T,l) 



— log|g_„^„ (t)|. 



The function / is a continuous function on [1, 2] by Lemma r4.2.5l To see this observe that by the 
Cauchy criterion the sum / converges uniformly for t E [1,2]. Since a uniformly convergent sum 
of continuous functions is continuous, the proposition is proved. □ 

Proof of lemma (|4.2.5|) 

Lemma 4.2.5. For s E [1, 2],p > w > 0, and v, u both not integers, t = x + iy, Im(T) > let 



h{w,s)= y 



[tot + n|2* 

u!<|mr+n|2<p 
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Then 

h{w,s)=o(wi-'''^ 

uniformly for s e [1,2]. 

The implied constant does not depend on u> or s. Our proof will work for s e (i, oo). However 
the interval [1,2] is sufficient for our application. 



Proof. Without loss of generality we assume t; ^ Z and Re(T) > 0. We estimate the sum 

\mT + n\ 



E I ^12. - (4-2-6) 

<|mr+nP<p 



Sum l|4.2.6l l is a sum over lattice points of A that are outside of the circle of radius ^/lo but inside 
the circle of radius y/p. It suffices to restrict our sum to the first quadrant of the plane since the 
sum over the other three quadrants can be estimated similarly. With this assumption n > and 
m > and our sum can be written as an explicit double sum with n and m separated as, 

[\AF/|r|] q{p) 27rmt, (VP/kll 9(p) 2-Kinv 

Eg277i(mM) _£ ^ g27ri(mn) _J: ,^ ^ 7) 

^ |mT + n|2s ^ /-^ |mT + n|2«' 

m=0 n=q{w) ' ' m=[V^/|T|] + l "=0 ' ' 

where q(a) = \\J a — wP-y^ — mx] . The brackets, [ ] represent the greatest integer function. The sum 
above is split over m. For in the first case horizontal translations of lattice points will intersect the 
inner circle \z\= ^fui and hence n must be restricted while the second sum n is free to start at zero. 
See figure l|4.2|l . We encourage the reader to write down the explicit double sum in the simple case 

of zezi. 

We estimate the inner sum using a Stieltjes integral. For t > let Tx(t) = [i] . Then for < a < 6 



h 



27Tinv 



where a' = a — S, b' = b + S for any S > sufficiently small (the Stieltjes integral must have a 
slightly larger integration domain to include the endpoints of the sum). Next using integration by 
parts with 

^ {{t + mx)^ + {my)'^Y ' 
dV{t) = e^""'-" dnit), 

V{t) = f e^"""'-" dn{x), 
Jo 



dUit) = ^Uit) dt, 
dt 



and noting that 



[t] 

V{t) 



•^0' k=0 
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n— horiziontal m— vertical 



Split off point for m, 
n varies^om to edge of big 
cijrfe 



n is restricted from 
inner circle to otiter circle 



Figure 4.2.1: Lattice points for the sum. 



satisfies —C<V{t)<C since v is not an integer, we can estimate the integral 



dirit) 



< 



u{t)vit)t 



+ 



\V{t)\ \dU{t)\ 



< c 



c 



\dU{t)\ 



< 2C 



U{t)t, <2C\U{a)\+2C\U{b)\. 



Note that \dU{t) \ is easily estimated since U{t) is monotone decreasing and that the primes can be 
dropped since 6 can be made arbitrarily small. 



What remains is to apply the above estimate to each of the inner sums and crudely put absolute 
values around all the other terms. 
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The first sum can be estimated as follows. 



[V^/\r\] 

E - 

m=0 



E 

n=q{w) 



Imr + 



[v^/|r|] 
< ^ ^ I g^Trimu I 

m=0 



E 

i=i}(ii)) 



1 2s 



= E 

[v^/|r|] 

E 

m=0 



9(P) 

E ■ 

l=q{w) 



\2s 



q{p)' g2-n-itu 



q{w) 



I \mT + n 



2s 



d7r(i) 



[v^/kl] 

<2C ^ (|C/(gH| 

m=0 



|C/(9(P))I) 



Simplifying U{q{w)) and U{q{p)) using 



t/(t) 



and 



yields 



[v^/|r|] 

2C ^ {\U{q{w))\ + \U{q{p))\) 

m=0 



m=0 

<2C[x/^/|t| + 1]( 



1 ^ 1 

wY [p] 
1 1 

w]^ [p] 



<2C[v^/|t| + 1]( 



1 



o 



The last inequality follows from w <p. 
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We apply the same method to the second sum and obtain, 

IVp/H] q{p) ^27r4«r 



„ ,mr + nr 

TO=[v^/|r|]+l n=o ' 

<2C ^ (|[/(0)| + |C/(g(p))|) 
m=[v^/|T|] + l 
[v^/kl] / ^ ^ 

- ^ i((ma;)2 + (my)2)« + 

rn=[\A«/|T|] + l 

[v^/kl] ^ 
" , ((mar)2 + (my)2)« + ^ 

?n=[\/w/|T|] + l m=[^/\T\] + l 

It remains to show that both sums above are bounded by a term of growth O {w^~^^ . 
For the right hand sum. 



[VP/\-r\] 

E 



2^ E |^^2C[v^/M]|^ = o(-'"0 



smce p> w. 

For the left hand sum 



E ((ma;)2 + {my)^y " E 

m=[v'«'/|'r|]+l m=[-yw] 

for some Ci > 0. The last equality holds xmiformly for s G [1, 2] by the integral test. □ 



4.3 The Explicit Trace 

In this section our main goal is to give an explicit formula for 

/ trv{Kr{P,P))dv{P). 

Following Selberg's original method we decompose 

Kr{P,Q) = Y,xil)K{P,^Q) 

into various sub-sums. Depending on their type. The types are as follows, "id" is the identity, "par" 
are the parabolic elements, "ce" are the cuspidal elliptic elements*, "nee" are the non-cuspidal 



^These elliptical elements share a common fixed point in P with some parabolic element in T. 
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elliptical elements, "lox" are the hyperbolic and loxodromic elements, and "cusp" — "par" U "ce". 
For each S G {id, par, ce, nee, lox, cusp} set 

K^{P,Q)= E x{l)KiP,jQ). 

Here denotes the subset of T consisting of elements of type S. 
Following IIEGM98[ Section 5.2, Theorem 6.5.1] we have. 



Lemma 4.3.1. 



/ try (/^j? + + K'°^) (P, P) dv{P) 

[ h{i+t^)t^dt+ y , , 

^« {fece 4|£:(i?)|sin^(4|y) 



':fa 

vol(r\H=')dimc(V) ,,2w2^, , V- tr(x(i?)).9(0)logiV(ro) 



, ^ tr(x(T))g(logiV(r))logjV(ro) , ^^^^^^ 

|£(T)||a(T)-«(T)-p +oil)asA^oo. (4.3.1) 



The notations above were defined in ^14. II 



The Cuspidal Elliptic Elements 

Our next immediate goal is to evaluate 

/ trvK'r%P,P)dv{P). 

JTa 

For notational simplicity, we will adopt Assumption 13.7.2] from this point on until the end of 
this thesis. 

Denote by C£ set of elements of T which are F-conjugate to an element of Too \ F^ = {76 
^00 I 7 is not parabolic nor the identity element }. We fix representatives of conjugacy classes of 
C£, (71, ... , gj^ that have the form 

"-C,:?')^ (4.3.) 

For g ^C£, letC(g) denote the centralizer in F of g. In addition, let {pi, 00} be the set of fixed points 
in P of the element gi . Since gi is a cuspidal elliptic element it follows that pi is a cusp of F (see 
IIEGM 98I page 52). Hence by Assumption 13 . 7.2l there is an element 7i € F with ^ioo = pi. Suppose 
that Ci is the lower left hand (matrix) entry of 7^. Then we have (see IEGM98t Pages 302-304]), 



'There are only finitely many distinct conjugacy classes of elliptic elements in a cofinite Kleinian group. 
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Lemma 4.3.2. 



[ ivKf'iP.P) dv{P) 



\C{9^)\ 



E 



2g(0)(log|c,|+logA) 



II 



^212 



e. I Jo 



5(2^)- 



sinh X 



cosh X — 1 



dx 



0(1) as A ^00. (4.3.3) 



4.4 The Parabolic Elements 

This section contains the new features of the Selberg trace formula that are not present in the 
two-dimensional vector and three-dimensional scalar cases. We remind the reader that Assump- 
tion |322] is in effect. 

Our main goal for this section is to evaluate 



/ tvvKl''\P,P)dv{P). 



(4.4.1) 



Evaluation of Integral 14.4.11 

Let The a fundamental domain for the action^" of Too on C, 

V={{z,r) eB.^ \ zeV}, 

and 

Va ={{z, r) gB.^ \ z eV, r < A}. 

It follows that is a fundamental domain for the action of Too on H'^. 

Recall that P^ is canonically isomorphic to a lattice Aqo- For /i G Aqo let /i denote the corre- 
sponding parabolic element in P^ . We will need the following (see l,EGM98. Pages 300-301]) 

Lemma 4.4.1. 

f ivKl'"'{P,P)dv{P)= ir{x{p)) f K{P,'pP)dv{P)+ o{l) . (4.4.2) 

Since P^ is an abelian group, x restricted to P^ can be diagonalized. In other words, there 
exist lattice characters {ilJi}i=i...n so that 



(4.4.3) 



1=1 



Thus it suffices to consider lattice characters instead of unitary representations. 



"See ^2.5l for more details on the action. 
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Lemma 4.4.2. Let ipbe a lattice character o/Aqo- Then 
(1) For ip — id, 



poo : T'J^ 
(2) For tP ^ id, 



HI) 



5(0) log A + '-^ + g{Q) - ^) _ ^ ^ + t')^{l + It) dt 



+ o(l) as A ^ CO. 



5(0) 



Proof. (1) is proved in I1EGM98I pages 300-302] 



— L(A, -0) + o(l) fls ^ — > oo. 



The proof of (2) is a modification of (1). Let Co, 



|A^| 



then by the definition of the action 



of /2 on H^, 

^' vx/i) / K{p, np) = Coo E' ^(A*) ^ (IS + 1) ^ ■ 



Since 



^ooE'#nfc(-+l)^- (4-4.4) 



(4.4.5) 



\^l\^<x 

using summation by parts, we can rewrite | |4.4.4|| as 

poo 

Coo / k{u + l)Z{2A'^u,A,tl))du. 
Jo 

Next we apply Proposition l4.2.2l to obtain 

/>oo 

Coo / k{u + l)Z{2A'^u,A,tP)du 
Jo 

= Coo^ k{u + l)(^L{A,'i(j)+o(^i2A^u)-^/^^^ du. (4.4.6) 

Now we show that the resulting error term is o(l). Since k{u + 1) = O ((1 + tt)^**) {k is a rapid 
decay function) 



/c(m + 1)o((2A2u)-i/2^ 



< 



(1 + u)~^u'^/^ du = o(l) as A ■ 



oo 



for some > 0. To complete the proof note that k{u + 1) du — g{0) . 



□ 
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Finally we can evaluate l|4.4.Hl : 
Lemma 4.4.3. 



(5(0) logA+^+ giO) {!l^-jyl.J^h{l + + ^t) dt 



|roo:r^l 



L{K^,rPi). (4.4.7) 



/ = /oo+l 



Here n = dime V, ipi are the lattice characters associated to the lattice Aoo , loo = dime > ^'^^ ^700 is the 
analogue of the Eider constant for the lattice Aqo- 

Proof. The proof follows immediately from Lemma 14.4.11 Equation l4.4.3[ and Lemma 14.4.21 □ 

We have evaluated the truncated trace of JC explicitly as an integral, and by using spectral 
theory. Notice that as A ^ 00 the integral over the parabolic sum (Lemma 14. 4.31 1 has a divergent 
term. So does the corresponding cuspidal elliptic integral (Lemma |4.3.2t . By Lemma 14.1.41 the 
divergent terms must equal the divergent term of the spectral (truncated) trace (Lemma 14. 1.711 . It 
follows that 

25(0) log A ^ |^(J;')^|^j^2|2 + 5(0) log A ^^J^^, , - g{Q)ko^ log A = 0. 



By choosing a suitable k so that (/(O) 7^ we obtain 
Lemma 4.4.4. 

^|C(g,)l|i-£fP |roo:r^| 

The formula^ ^ above is an application of spectral theory to the group relations of a cofinite 
hyperbolic three-orbifold^^. We will use the above lemma to give a meromorphic continuation of 
the Selberg zeta-function. 



Completion of the proof of the Selberg Trace Formula 

The Selberg trace formula now follows: combine Lemma |4.4.3[ Lemma |4.3.21 Lemma l4.4.4l Lemma |4.3.1[ 
Lemma I4.1.7i and Lemma 14.1.41 Note that the divergent terms all cancel by Lemma 14.1.41 (or we 
can use Lemma [4.4.41 1. Finally take the limit as A — > 00. See IEGM981 Section 6.5] for more details 
on combining the lemmas above. 



A similar formula is valid for the general case of K-many cusps. 

'Notice that all of the terms above are defined simply in terms of group relations. 



Chapter 5 

The Selberg Zeta Function 



In this section we define the Selberg zeta-function Z{s, T, x) for cofinite Kleinian groups with 
finite-dimensional unitary representations, in the right half -plane Re(s) > 1. We then evaluate the 
logarithmic derivative of Z{s, T, x) and show that Z{s, F, x) admits a meromorphic continuation, 
subject to some technical assumptions concerning the stabilizer subgroup Foo ■ 

5.1 The Definition and Motivation 

In the celebrated paper [Sel56'| Selberg first defined what is now called "The Selberg zeta-function^" 
as an infinite product over lengths of primitive dosed geodesies^, bearing a strong resemblance to 
the Riemann zeta-function. Surprisingly, the Selberg zeta-function satisfies a Riemann hypothosis, 
and encodes both geometric and spectral data of the quotient orbifold^ r\H ^. The spectral and ge- 
ometric connection is made clear when one understands the Selberg zeta-function as a by-product 
of the Selberg trace formula applied to the resolvent kernel of A . 

In Def intion 15 . 1 . 1 1 we will define the Selberg zeta-function for our case of interest. A natural 
question arises: what does our zeta-function have in common with the original Selberg zeta-function? The 
answer*: the logarithmic derivatives of both zeta-f unctions are directly related to the loxodromic 
(or hyperbolic) contribution of the Selberg trace formula applied to the resolvent kernel of A . The 
term (from the trace formula) in question for our case has the form 



We show in Lemma 15.1.21 that the term above is the logarithmic derivatives of a meromorphic 
function Z{s, T, x), and that it has a product expansion in the right half-plane R{s) > 1. 

^More precisely, the Selberg zeta-function of a cocompact Fuchsian group. 

^Geodesies that do not trace over themselves multiple times. 

Riemann surface if F is torsion-free. 

*An alternative answer is that in the cocompact case, both zeta functions are factors of the regularized (functional) 
determenant dot (zi — (1 — s^)) . See ISar87l for more details. 
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In order to define Z{s,r,x) we will need some notions concerning centralizer subgroups of 
loxodromic elements. For more details see llEGM98i Sections 5.2,5.4]. 

Let r be a cofinite Klernian group and let x S Rep(r, V) . Suppose T e F is loxodromic (we 
consider hyperbolic elements as loxodromic elements). Then T is conjugate in PSL(2,C) to a 
vmique element of the form 

Dm - ( ""(^^ ° 

such that a{T) e C has \a{T)\ > 1. Let N{T) denote the norm of T, defined by 

7V(r)^|a(r)p, 

and let by C{T) denote the centralizer of T in F. There exists a (primitive) loxodromic element To, 
and a finite cyclic elliptic subgroup £{T) of order m{T), generated by an element Et„ such that 

C{T) = (To) X £{T) . 

Here (Tq) = { Tq" | n e Z }.^ Next, Let ti, . . . , t„, and t'^, . . . , t^^ denote the eigenvalues of x(To) and 
x(i?To) respectively. The elliptic element Eto is conjugate in PSL(2, C) to an element of the form 

an) 

C(To)-i 

where here C(2o) is a primitive 2TO(T)-th root of unity. 

Definition 5.1.1. For Re(s) > 1 the Selberg zeta-function Z{s, F, x) is defined by 

dim V 

Z{s,r,x)^ n n n {l-iMTo)-''^(ToF^N{To)-^-'). 

{To}G7^ j=l l,k>0 

c{T,],Lk) = l 

Here the product with respect to Tq extends over a maximal reduced system TZ of F-conjugacy 
classes of primitive loxodromic elements of F. The system TZ is called reduced if no two of its 
elements have representatives with the same centralizer^. The fimction c(T, j, k) is defined by 

c(T,j- ;,fc) = t;c(To)2'C(7o)-'^ 

Lemma 5.1.2. f or Rc(s) > 1, 

ds ^^^^^m{T)\a{T) - a{T) 



^Note that by definition £{T) = £{Tq), and that To is unique up to multiplication by an element of £{T). 
'■See IEGM98[ section 5.4 for more details 
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Proof. It follows from the proof of IIEGM98[ Lemma 5.4.2] that 

^ tr(x(T))logjV(To) 



{To}eK 
n>0 
l<v<m{To) 



m(ro) |C(ro)-a(ro)"+i - C(To)-a(To)-"-ir 



Next since Tq cormnutes with Etq we can diagonalize the restriction of x to C (T) and continue the 
equality to 

^ t"+it'"log7V(To) , 

= y ' ^ ^ ^iV(ro)-"("+^) 

{T^T^ m(ro)|C(ro)''a(ro)-+i-C(To)-''a(To) — 

n>0 



l<v<m{To) 
l<i<dim V 



^ t"+4'"log7V(To) , 

^ (To)(i-d(ro))(i-d(ro)) 



{To}e7?, "1 
n>0 
l<v<m{Ta) 
l<j<dim y 



t^"+it';7V(ro)-^("+i) log7V(To) (d(ro))'- (d(To)) 
^ m(To) 

n>0 
l<«<m(To) 

Z,fe>0 
l<j<dim V 



where 



d{To) = C(ro)-''^a(To)-2("+i) 

Next we sum over the u— index (note that it is a geometric sum of an m(To)— th root of unity) 
observe that the sum is non-zero only when 

t',c(To)2'c(ro)-2fc = 1 

or using our notation c(T, j, l,k) = 1. The equality continues as 

= y t^"+iiV(To)-^("+i)iogiv(ro) (a(ro)-2(«+i))' (a(ro)-2(«+i) 

{ro}G7?. 
n>0 
i,fc>0 
l<j<dim V 
c(T,j,i,fc) = l 



V- t,-a(To)-^fca(ro)-2'jV(ro)-(-+i)logjV(ro) _ Z'i.s,r,x) 

{r^T^ 1 - %a(ro)-2fca(ro)-2;7v(To)-(^+i) ^(s,r,x). 

i,fc>0 
l<j<dim V 
ciT,j,l,k) = l 
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□ 



5.2 The Logarithmic Derivative of the Selberg Zeta-Function 

The first step in obtaining the meromorphic continuation of the zeta-function is to relate its loga- 
rithmic derivative to the trace formula. From this point on Assumption l3.7.2l is in effect. 
We apply the Selberg trace formula to the pair of functions, 

s^ + iy— 1 iJ^ + w — 1 

where 1 < Re(s) < Re(B) and obtain 
Lemma 5.2.1. 

1^(A^±^(B) = 1- V tr(x(T))logjV(To) 

2s Z^' 2BZ^ ' 2s ^^^^m(T)|a(r) -a(r)-i|2 ^ ' 

_J_ ^ tr(x(T))logjV(To) 
1 \ 1 



^^\s^- si - si) 47r j^\s^ + w^ B^ + J (j) ^'"^^ '^'^ 

loo f f ^ 1 \ r', , , tr6(0) trS(O) 

' ' ^ {l + iw)dw+ ^ ' ^ ' 



27r[roo : ri^] Jr\s^ B^ + J T ^ ' 4s2 4^2 

V- tryx(i?)log^(ro) vol(r\H3)dimF _ 
2s 2BJ 4|£(i?)|sin2(^) An ' 

1 ^ ^ ^ 2trx(ffi) logical 



2s 2B 



1 1 



2s 2B)\r^:rj, 



0(^-7)+ E i(Aoo,V/)). (5.2.1) 



Proof. The first equality follows from Lemma [5. 1.21 The second equality follows directly from the 
Selberg trace formula. □ 

Equation 115.2. It is used to exhibit the meromorphic continuation of Z{s, T, x). If we fix B and 
multiply through by 2s, it is not hard to see that each term on the right of (|5.2.1l l is meromorphic. 



5.2. THE LOGARITHMIC DERIVATIVE OF THE SELBERG ZETA-FUNCTION 



63 



In order to see that Z{s, F, x) is meromorphic, we must compute the residues of each term on the 
right of l|5. 2.111 . We will show that the residues are fractional and that for some N E'N, Z(s, F, x)^ 
is a meromorphic function. 

Theorem 5.2.2. Let F be cofinite with one class of cusps at ( = oo, and let x G Rep(F, V) . 
1- lf[roo ■ r^] — lor [Fryo ■ F^] = 2, then Z{s, F, x) is a meromorphic function. 

2. If[Fao ■ F^] = 3, then there exists a natural number N, 1 < N < 6, so that (Z(s,F,x))^ a 
meromorphic function. 

Proof. The proof follows from a careful study of 1 15.2. It . We must show that after multiplying by 2s, 
each term on the right (of the second equal sign) has at most simple poles with integral or rational 
residues''. This is demonstrated in Lemma l5.2.4[ Lemma l5.3.1[ Lemma l5.3.2l and Lemma l5.3.4l □ 



We remark that the divisor of the Selberg zeta-f unction is readily read off from Lemma 15.2.41 
Lemma [5.3.1[ Lemma [5.3. 2i and Lemma [5.3.41 

Our zeta function satisfies a functional equation. A standard argument ( IVen82[ Theorem 5.1.5, 
page 85]) using (|5. 2.111 , Lemma [5.2.4[ Lemma [5.3.1[ Lemma [5.3.21 and Lemma [5.3.4l yields: 

Theorem 5.2.3. Suppose that [i"oo : F^] = 1 or [Faa : F^] — 2. Then Z{s, F, x) satisfies: 

Z{-s, F, x) = Z{s, F, x^{s)-^[s, F, x)- 



For [F^ : r^] = 1, 



and 



,'F(l-s)\''°° / vol(F\H3)dimy ^ ^ 



For \F^ : r;,] = 2 



F(l 



F(l + s) 



nexp(-fc(-l)' 



(fc + 1)2 - s2 



• exp 



vol (F \ M?) dim V 



Es + C 



^For case (1) the residues must be integer while for case (2) it suffices to show that the residues are rational with 
bounded denominator. 
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and 



tryx(i?)logiV(ro) , ^ 2trx(50 logic. 



1 / " 



T/je constant C satisfies the equation: exp(C) = ±1. 
Lemma 5.2.4. The expression 



neD 



has only simple poles and integral residues: 

(a) at the points ±Sj on the line Re(s) ~ and on the interval [—1, 1]. Each point Sj is related to an 
eigenvalue Xj of the discrete spectrum of A by 1 — = Aj . The residue of each Sj is equal to the multiplicity 
of the corresponding eigenvalue. If X = 1, is an eigenvalue of A, then the residue of the point sj = 0, is 
twice the multiplicity ofX; 

(b) at the points pj that are poles o/S(s), which lie in the half-plane Re(s) < 0. The residue of each pj 
is non-negative'^ and equal to its multiplicity as a pole o/6(s). 

Proof. The computation involves elementary complex analysis. See IIVen82[ Section 5.1]. □ 

The residues above come from terms that are related to the spectral and scattering theory of 
A . The remaining residues are computed using group theoretic data involving F and x- The poles 
and zeros of Z(s, F, x) that correspond to these residues are commonly called topological or triviaf^. 

5.3 The Topological Zeros and Poles 

The computation of the topological residues is considerably more complicated than the corre- 
sponding spectral computation. Poles can only arise from the following terms (excluding the 
spectral terms previously dealt with) of 115.2.11 (note that we multiplied all terms through by 2s): 

^oo f 2s F' l^ , tr6(0) 

' "(1 + "iw) dw ' 



da;. (5.3.1) 



trx(gi) f°° „-sx sinh: 



^|C(g.)l|l-e?Pi„ coshx-l + ^^ 



^The value of C can be read off by letting s in the functional equations. Its value depends on whether 0(0) is 1 or 
— 1 and the multiplicity of Z{s, F, x) at s = 0. 

'The point pj is a zero of Z{s, T, x) and a pole of S(s). We understand the multiplicity of a pole as non-negative 
number (not as a negative number). 

iO\Ye refer to them as topological. 



5.3. THE TOPOLOGICAL ZEROS AND POLES 
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The first two terms come from the parabolic elements of T, the third from the spectral trace, 
and the last from the cuspidal elliptic elements of F. It is remarkable that the last three terms need 
to be taken together in order to compute the residue at s = 0, while the first and last are needed to 
compute the residues on the negative real axis. 

It is well known that 



3— + 



In order to obtain a similarly explicit formula for 

trx(g») f°° -sx sinhx 
.9^)\\l-el?J 

we must make some technical assumptions. 



y^ '\tr / — (5.3.3) 



Case One: [F^ ■.r'^\ = l 

In this case, I l5.3.3t is not applicable and loo = k^o = k{T,x) (the last equality follows from our 
assumption that oo is the only cusp). Since S(0) is a unitary self-adjoint matrix of dimension kx k, 
its trace consists of a sum of k terms of the form ±1. It follows that ■i(trS(0) — k) is an integer. 
After applying equations 1 15.3. Il l and | |5.3.2| | we have: 

Lemma 5.3.1. Suppose [Too ■ r^] = 1. Then the poles of 1 15.3. Ijl are simple and are located at the points 
s = —1, —2, . . . , with residue k^o and at the point s = 0, with residue i(tr 6(0) — koo). 

Case Two: [F^ : T^,] = 2 

In this case for all i, ei = e = and l|5.3.3l l becomes 

^|C(50l|l-^Pio cosh:. + l ^^■^■^> 

An application of Lemma [4.4.41 gives us the coefficient of the integral above 

Next, to evaluate the integral in | |5.3.4| | we appeal to the following formula: 

r 6-^"= {cosh X- cos ty^ dx^—Y^^, (5.3.5) 
Jo sintf^^ti + k' 
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valid for Re(/^) > — 1 and t ^ 2mT, (see IIGR65I formula 3.543.2). Averaging for the two values 
/i — 1 , /i + 1 , we obtain 

Binhx ax=^y sin kt ( ^--) . (5.3.6) 

Jr. cosli X — COS t sm t Vs^l + fi^ s + l + fc/ 

k—l 

Finally, we can evaluate the integral in l|5.3.41 by taking the limit ast ^ tt 

r e-^^^ dx^y fc(-l)^+i ( ^--] . (5.3.7) 

Jo coshx + 1 ^ ^ ' Vs-l + fc s + l + k) ^ ' 

Combining the residues above with those from 115.3. 2b give us: 

Lemma 5.3.2. Suppose [Fao '■ r^] — 2. Then the poles of l |5. 3.111 are simple and are located at the points 
s — n, with n = — 1, —2, . . . , and residue^^: 

{fcoo, if n is odd, 

Iqq koQ, else 

and at the point s = with residue i(tr 6(0) — fcoo)- 

Since the Picard group V = PSL(2, Z[x/^) satisfies [F^ : I^] 2 we have: 

Corollary 5.3.3. Let T = PSL(2, 'Ly~l]), and let x e Rcp(r, V) . Then Z{s, P, x) is a meromorphic 
function. 

Case Three: [Foo : T^,] = 3 

In this case, the cuspidal elliptic elements gi that are in I l5.3.3b must all be of order three. Hence 
ei e {v^, v^— 1 }, but when such is plugged into |1 — e^p one obtains 3. Thus we can rewrite 
(|5331 l as 

' trx(5*) f°° -sx sinha; 



y '\::r / e- ~, dx. (5.3.8) 

Next, applying 1 15.3. 6b with t = ^tt, and Lemma [4.4.4[ we can rewrite l|5.3.8b as 

1 /, ?oo\ // 1 1 \ / 1 1 



2V''°° 3/VVs-l + l s + l + iy \.s-l + 2 s + 1 + 2^ 

1 1 \ / 1 1 



s-1 + 4 s + 1 + 4/ Vs-1 + 5 .s + 1 + 5 
Finally combining the residues above with the residues from (|5.3.2b we obtain: 



(5.3.9) 



^^Note that by defintion ioo > fco 



5.4. THE ENTIRE FUNCTION ASSOCIATED TO THE SELBERG ZETA-FUNCTION 
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Lemma 5.3.4. Suppose [Foa : F^] ~ 3. Then the poles of l|5.3.Hl are simple and are located at the points 
s — n, with rt = — 1,— 2,..., and residue: 

— fcoo , ifn is a multiple of 3, 
+ 5^00, else 

and at the point s = with residue i(tr 6(0) — fcoo)- 
As an application of the lemma above, we have: 

Corollary 5.3.5. Let T = PSL(2, Z[-i + ^]), andx=l (the trivial representation). Then Z{s, T, x) 
is not a meromorphic function^^ (it is the 6-th root of a meromorphic function). 

Proof Since Q(\/^) has class number one, PSL(2, Z[-i + ^]) has one class of cusps ( IIEGM98i 
Chapter 7]). In addition it has oo for a cusp. Elementary calculations show that [Foe ■ F^] = 3. By 
definition, x = 1 implies that fcoo = 'oo = 1- The result follows from Lemma 15.3.41 □ 



The Remaining Cases 

The cases that remain are: [i^oo ■ = 4 and [I^oc ■ ~ 6. Using the same ideas as in the other 
three cases one can show the following: 

Theorem 5.3.6. Suppose that [Foo ■ F^] — 4. Then for some integer N, Z{s,T,x)'^ is a meromorphic 
function. 

The author does not know a good bound for the integer N {N depends on F and x)- On the 
other hand we conjecture the following: 

Conjecture. Suppose that [F^o ■ F^] — 6. Then for some integer N, Z{s,T,x)^ is a meromorphic 
function. 



5.4 The Entire Function Associated to the Selberg Zeta-Function 

As Fischer [Fis87', Chapter 3] observed, it is useful to group the elliptic, parabolic, and identity 
terms, together with the loxodromic terms to define an entire function associated to the Selberg 
zeta-function called the Selberg xi-function S(s, F, x)- 




i^This is the first example that the author is aware of where the Selberg zeta-function is not meromorphic. 
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Definition 5.4.1. For Rc(s) > 1 set 

=L( r ^- \- tr(x(T))logjV(To) 



(1 + iw) aw 



27r[r^:r^]J^\s^+wy T 
^ tr6(0)- 



trx(g») /-""/ e-^^ N sinhx _ 

it |C(3.)l|l-efPyo I 2s ;eoshx-l + ^ ' 

where E is defined in Theorem |5.2.3l 

Theorem 5.4.2. The Selberg xi-function can he continued to an entire function S(s, F, x) with 

^|(.,r,x)-^|(s,r,x) 

= Yl ( «2 _ «2 " 52 _ ^2 ) ^A^j^ ( s2 + y;2 - B'^+W'^l '^'^ ' 



HereB > 1, B > Re(s). 



Proof. The second assertion follows from ( I5.2.1I I. For the first asserstion: by Lemma I5.2.4I after 
multiplying through by 2s, the residues of the expression on the right side of the equal sign are all 
integers and positive. Hence S(s, F, x) is entire. □ 

An explicit product formula (for Re(s) > 1) can be obtained for S(s, F, x) in the case of [-Too : P^] — 1,2,3 
by integrating, and then exponentiating, the explicit formula for the contribution of the parabolic 
and cuspidal elliptic elements to the Selberg zeta-function. 

Remark 5.4.3. The Selberg zeta-function for symmetric spaces of rank-one was studied hy Gangolli and 
Warner ( I.Gan77h iGWSOi ). More specifically, they studied torsion-free cocompact quotients with finite- 
dimensional unitary representations, and non-cocompact torsion-free quotients with trivial representations 
(the scalar case). 



'Theorem l5.4.2l shows that we are justified in defining H{s, F, x) by its logarithmic derivative. 
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